3.2 Amazon X-Ray
5.1.5 User Authentication
Authenticating users to ensure a user is who they say they are is a critically important part of any web application. AggrGator allows new users to create accounts and specify a desired username and password upon creating a new account, and uses Passport.jsfor user authentication when users log into the website. A custom local strategy was written for AggrGator’s user authentication mechanism. All web pages listed in Table 2 above that have "Login Required" marked as "Y" are protected and required users to login to AggrGator before they can visit those webpages; users can log into the application (or conversely sign out) from any page. Logged in users have access to a drop-down menu that appears when they click on their name in the upper right hand corner of the page.
Fig. 5.5: (AggrGator login bar):all pages on the site are available for users to log in on
Once a user successfully logs in, there are three new items that they can then interact with on the site:
Fig. 5.6: (AggrGator user authentication failure): invalid user login attempts will see this view
Fig. 5.7: (AggrGator successful login):successful login attempt for user "dkilleffer"; note that now there is a site-wide search bar exposed at the top, the user’s name is shown (and site navigation menu visible when clicked on), and the "Upload Media" link button in the upper right hand corner
• a search bar at the top of the page which lets users conduct free-text searches across all media
• custom link menu and user name (when the user clicks on their name a drop-down navigation menu appears)
• an "Upload Media" button in the upper right hand corner of the page
The custom local strategyPassport.jsimplementation usesElasticsearchfor storing user information and password hashes. When a new user account is created, the User class uses aJavaScript bcrypt.jslibrary to create a password hash, and that
Fig. 5.8: (AggrGator logout):displayed upon successful user account logout
hash is stored alongside the user account metadata inElasticsearch. Once a user logs in and they have been authenticated against the password hash that is stored in Elasticsearch, a new cookie based session is created for the user and stored on their filesystem as a cookie. SubsequentHyperText Transfer Protocol (HTTP) requests after the initial login update the default timeout value (15 minutes - set in
prototype.yamlasprototype:session:maxAge) update the last checked time, so as long as logged-in users are continually loading pages, they remain authenticated and logged in (as soon as a logged in user has over 15 minutes of inactivity they will be logged out on the next page request they make).
5.1.6
Domain Model
AggrGator follows a common design pattern (Model View Controller) to implement the user interface, and the domain model contains the part of the codebase which contains much of the business logic. By splitting the codebase up and separating the files which comprise the "views" (e.g., the user interface), the "controllers", and the "models", it is easier to make changes to the models without potentially introducing
problems in the views or controllers and make incremental changes. The models located under the/lib/models/directory are:
• Annotation.js:models what an annotation object stored in Elasticsearch looks like
• Image.js: models an image stored in Elasticsearch; extracts metadata from image objects and so that data can be stored in Elasticsearch; child class of Media
• Media.jssuperclass of Image and Video; contains the top-level properties for all media objects; has methods for extracting metadata of Images and Videos • User.jsmodels an AggrGator user account; contains user password authentica-
tion methods
• Video.jschild class of Media, contains properties specific to Videos; has meth- ods to extract thumbnail images from Videos
Due to the prototypal nature ofJavaScriptobjects, the massive benefit of separating the models from the other parts of the application is that once an object is created, it can easily be stored inElasticsearchby simply passing the objects to the controller class,lib/controllers/ElasticsearchAPI.js.
The Media class is a parent class of both Image and Video, and provides some functions common to both. When a user uploads an image or video file, several sequential operations need to take place, that are slightly different depending on which type of item is being uploaded. The table below lists what actions are taken on the server side by the domain model code when a user uploads a media item (whether an image or a video):
Sequence Order Image Uploads Video Uploads
1 user attempts to upload a
file, andonChunkedUpload()
method in app.js is called to handle the file upload, then
Media.detectMediaType() is called to determine whether the file is image or video; also detected is whether or not the upload came from the user pro- file page, so if the file upload is an image it should be consid- ered a "profile image"
user attempts to upload a file, andonChunkedUpload()
method in app.js is called to handle the file upload, then
Media.detectMediaType() is called to determine whether the file is image or video
2 Media.js setOwner() called
and current user info passed to assign owner properties on media
Media.js setOwner() called and current user info passed to assign owner properties on media
3 Image.js constructor called to create new image object
Video.js constructor called to create new video object 4 Image.js init() called to get
and set image file metadata
Video.jsinit()called to get and set video file metadata
5 Media.js getFileMetadata()
called to get file size informa- tion (including filesize in bytes, filesize user display value, file creation time)
Media.js getFileMetadata()
called to get file size informa- tion (including filesize in bytes, filesize user display value, file creation time)
6 new JavaScript Promise cre-
ated to extract EXIF image metadata from the image, in- cluding width, height, orienta- tion, file path, etc. and then those attributes are set on the Image object
new JavaScript Promise cre- ated to extract video metadata from the video file, including width, height, running time, codec, etc. and then those properties are set on the Video object
continued on next page
Tab. 5.3AggrGator media upload backend processing sequence – continued from previous page Sequence Order Image Uploads Video Uploads
7 image file aspect ratio is calcu- lated by dividing original Im- age height by width (used to calculate how to scale a smaller thumbnail-sized version)
Media.js calculateThumb- nailDimensions() called to calculate the size a frame capture thumbnail image (tinyThumbnailWidth, tinyThumbnailHeight) and gallery sized image (thumb- nailWidth, thumbnailHeight) should be, while preserving the original video aspect ratio
8 a new thumbnail sized image
version of the original image is created, and properties for that thumbnail sized image set on the Image object
a thumbnail sized frame cap- ture image is created from the first frame of the video, and the properties of that thumb- nail image are set on the Video object
9 - a gallery sized frame capture
image is created (used for the coverflow display of videos on the "My Media" page) from the first frame of the video, and the properties of that gallery image are set on the Video object 10 the Image object is recorded
as a new entry inElasticsearch
and associated with the file owner
the Video object is recorded as a new entry in Elasticsearch
and associated with the file owner
continued on next page
Tab. 5.3AggrGator media upload backend processing sequence – continued from previous page Sequence Order Image Uploads Video Uploads
11 if the Image file upload was de- termined to be a user profile image (back in step 1), then several attributes of the owner User object are updated to re- flect that the owner’s profile im- age was added/updated, and then the User object is updated atElasticsearch
-
12 successful Image upload record is logged, user redirected to user profile page
successful Video upload record is logged, user redirected to video annotation page
Tab. 5.3: AggrGator media upload backend processing sequence
5.2
Controllers
In classicalModel View Controller design pattern implementations, the controller layer of the pattern is generally meant to be a distinct, discrete layer from the model and view layers of an application. In modernNode.jsandExpress.jsweb applications, there will often be a routing layer which handles user webpage requests and handles coordinating actions between presenting the view layer and interactions with the model layer. This routing layer will typically coordinate handling user-passed information (in the form of form submission values,HTTPGET parameter handling, etc.) along with model code to orchestrate the displaying of views to the user based on their input and interaction with the application. AggrGator follows this same architectural approach with regards to the controller layer, but also uses another design pattern called Data Access Object. The /controllers/ElasticsearchAPI.js
class acts as aData Access Objectfor the application and orchestrates all interactions with theElasticsearchdata persistence layer.
When starting the AggrGator application, the Node.js runtime will execute the
/bin/wwwfile, which performs some minimal port setup, reads inapp.js, and then creates an application server, listening to userHTTPrequests on the default port (3000). Inside app.js is where the majority of the application setup is declared and loaded, including declaring the routes that are used by AggrGator. All of the routes configured in AggrGator are declared byrequirestatements that load in the contents of theJavaScriptfile that each statement loads in - additionally all of the routes are passed apassportobject which is used to handle serializing/deserializing user sessions, as well as having methods for creating new user accounts (and creating a session cookie) and authenticating user accounts upon login. Since the passport object is passed to all routes, they all have access to thepassportobject’s authentication mechanisms, and thus each route can selectively declare which routes (accessible viaHTTPGET or POST) should require the user to be logged in or not.
All of the routes make use ofHogan.jstemplate views for the display that is presented to the user. The routes collect all of the information that they need to render the views to the user, and execute the business logic of retrieving information from the domain model (via theElasticsearchAPI.jscontroller and the various model classes), and then combine those elements byrenderingtheHogan.jsview. Each route declares an object calledrenderDatawhich is a simple JavaScript object but contains several attributes that eachHogan.js template uses to render the views.
Hogan.js supports a feature called partials, which allows the author to declare smaller, incomplete sections ofHTMLcode and then combine variouspartialsinto a cohesive completeHTMLwebpage. This allows AggrGator to define common design and layout elements that can be shared across all of the web pages that comprise the user interface, such as having a common footer (/views/partials/footer.html) and a common header (/views/partials/header.html), as well as several others. Hogan.js files are intended to be "logic-less" templates, and so the work of determining what should be displayed to the user is confined to the actual router files. The
93 // downstream endpoints will now have access to these properties in
the req object
94 app . locals . configuration = configuration ; 95 app . locals . logger = logger ;
96 app . locals . elasticsearchAPI = elasticsearchAPI ; 97 app . locals . messages = messages ;
98 app . locals . mainPageTitle = configuration . configValues . get ('prototype :
mainTitle ');
99 app . locals . getAppTagline = () => {
100 let taglines = configuration . configValues . get ('prototype : appTaglines ');
101 r e t u r n taglines [ Math . floor ( Math . random () * taglines . length )];
102 }; 103
104 // initialize passport for user authentication
105 let initPassport = require ( appRoot + '/ lib / passport / init ');
106 initPassport ( passport , elasticsearchAPI ); 107
108
109 // declare route handlers
110 c o n s t homepage = require ('./ routes / index ')( passport );
111 c o n s t howItWorks = require ('./ routes /how -it - works ')( passport );
112 c o n s t annotate = require ('./ routes / annotate ')( passport );
113 c o n s t annotations = require ('./ routes / annotations ')( passport );
114 c o n s t collaborate = require ('./ routes / collaborate ')( passport );
115 c o n s t about = require ('./ routes / about ')( passport );
116 c o n s t signup = require ('./ routes / signup ')( passport );
117 c o n s t login = require ('./ routes / login ')( passport );
118 c o n s t logout = require ('./ routes / logout ')( passport );
119 c o n s t searchMedia = require ('./ routes / search - media ')( passport );
120 c o n s t searchUsers = require ('./ routes / search - users ')( passport );
121 c o n s t profile = require ('./ routes / profile ')( passport );
122 c o n s t uploadMedia = require ('./ routes / upload - media ')( passport );
123 c o n s t annotateMedia = require ('./ routes / annotate - media ')( passport );
124 c o n s t myMedia = require ('./ routes /my - media ')( passport );
125
126 // maps routes to the route handlers 127 app . use ('/', homepage );
128 app . use ('/how -it - works ', howItWorks ); 129 app . use ('/ annotate ', annotate );
130 app . use ('/ annotations ', annotations ); 131 app . use ('/ collaborate ', collaborate ); 132 app . use ('/ about ', about );
133 app . use ('/ signup ', signup ); 134 app . use ('/ login ', login ); 135 app . use ('/ logout ', logout );
136 app . use ('/ search - media ', searchMedia ); 137 app . use ('/ search - users ', searchUsers ); 138 app . use ('/ profile ', profile );
139 app . use ('/ upload - media ', uploadMedia ); 140 app . use ('/ annotate - media ', annotateMedia ); 141 app . use ('/my - media ', myMedia );
142 app . post ('/ uploads ', onUpload );
Fig. 5.9: (app.js, lines 93-142) -passport user authentication declaration, route declarations
ElasticsearchAPI.jscontroller provides the following public methods that are used by various routers in the AggrGator application:
1. connect(connectOptions): opens a connection to theElasticsearchcluster. 2. loginUser(params): attempts to verify that a user with the supplied creden-
tials exists in theElasticsearchcluster; if a matching user is found, constructs a newUserobject and returns it; if no matching username/password is found, returns null
3. retrieveUser(params): attempts to look up a user in theElasticsearchcluster based upon the supplied search criteria; if a matching user is found, constructs a newUserobject and returns it; if no matching user is found, returns null. Dif- ferent thanloginUser(params)because user properties are what are matched against, not user credentials. Will only return a singleUserif a match is found. 4. retrieveUsers(params): queries the Elasticsearch cluster for all users that
match the supplied search parameters, and return an array ofUserobjects that match all those users found, or return an empty array if no matches were found.
5. addUser(user): will attempt to add a newUserto theElasticsearchcluster, but first will callretrieveUser(params)and pass the supplied email and username properties from theUserobject to ensure theUserbeing added is a new one. Returns the newUserobject on success.
6. updateUser(user): attempts to update an existingUserin the Elasticsearch cluster with the updated properties on theuserobject. Returns the updated
Userobject on success.
7. deleteUser(user): attempts to delete an existing User in the Elasticsearch cluster that matches all the properties on the passeduserobject. Returns null on success.
8. retrieveMedia(params): attempts to look up all media objects in theElastic- searchcluster that match the supplied query parameters. Based on the type of objects retrieved from the database, will either construct newVideo,Image, or
Mediaobjects and returns a single matching object (primarily used to retrieve media objects by unique values, such as ID, UUID, etc.)
9. removeUnindexedFields(indexObj): objects such as Media, Image, Video, and User may have field properties on them required for things such as logging, etc., which should not be indexed in theElasticsearchcluster, so this function will remove any such properties that are not known to the defined Elasticsearchmapping so that theindexObjcan be written to theElasticsearch index and not contain any unknown properties.
10. addMedia(media, owner): attempts to add a new media item to the Elas- ticsearchcluster; verifies that themediapassed is either aMedia,Image, or
Videoobject and that theowneris a validUserobject, and also removes all unindexed fields from both the mediaand user objects (by calling remove- UnindexedFields()on both objects) prior to attempting to add the item to Elasticsearch. Sets the media owner properties on themediaobject based on the passeduserobject, and then adds it to the database. Returns a specific error message if there is already a media item with any of the same unique param- eters (such asuuid,mediaPath,thumbnailPath, ortinyThumbnailPath). Will return the updatedmediaobject that was added to the database on success. 11. updateMedia(media, updater): attempts to update an existing media item in
theElasticsearchdatabase, and updates theupdatedproperty on the document to have the current time. Returns the updatedmediaobject on success. 12. deleteMedia(media, deleter): attempts to delete an existing media document
from theElasticsearchcluster. Returns null on success.
13. getMediaForOwner(owner, searchTerm): attempts to retrieve all media doc- uments from Elasticsearch for the passed owner that match the supplied searchTerm. If searchTerm is null, will try to match any media object that the user owns (but excluding any existingprofileImagedocuments). Returns an array of all matching objects (coerced into eitherMedia,Image, orVideo
objects based on theirmediaTypeproperty).
14. createAnnotation(annotation, owner): attempts to add a newAnnotation
to theElasticsearchcluster for theowner. Validates that theannotationis valid, then searches for a matching media object (whether anImageor aVideo) in order to append theannotationonto the respective property of the matching media object based on the annotation type. Finally updates the matching media object and adds onto the appropriate annotation marker (allPeople,allPlaces, allDates, orallTags), and returns the updated media object on success.
15. searchMedia(params): queries theElasticsearchcluster for all media docu- ments matching the supplied query parameters and returns the JSON result returned by Elasticsearch without modification. Will only return a singleMe- diaitem if the matching document was found inElasticsearch, otherwise will return null.
5.2.1
Code Documentation
Similar toJavaDoc,JShintis aJavaScript-based tool for automatically generating code documentation web pages based on the source code following particular com- menting and annotation rules. Before code can be checked into the source control repository at GitHub, the code checkin process automatically createsJShintcode documentation web pages and stores them under the/public/jsdoc/directory.
5.2.2
Automated Tests
During active development an automated test suite was built and utilized to ensure that the backendElasticsearchdatastore would function as expected, and that the domain model would also work as desired. Since the codebase heavily relies upon ECMA Script 2016 (ES6)Promises and asynchronous programming practices, the test suite was instrumental in ensuring that all effects of calling an asynchronous method would be executed. Before code could be checked into the version control
Fig. 5.10: (AggrGator JSHint code documentation):automatically created ElasticsearchAPI.js code documentation page
repository, a "pre-commit" yarn build script target would enforce that the "test" target would run, which executed all the test files under the top-leveltests/directory.
Early versions of the automated test libraries would utilizeJavaScript Object Notation (JSON)files with hard-coded objects (such as Media files, or Users), read thoseJSON files in upon the test harness loading, and then iterate over all theJSONobjects and