NGAGE is for measuring and driving wide-scale User Adoption of SharePoint 2013 and for continuous improvement in the quality of User Experience.
It also delivers extensive Site Traffic and Search Analytics - while serving as an auditing resource for Governance, Risk and Compliance requirements. It can also be extended to track usage of Yammer, HP WorkSite and virtually any Custom or Third Party application.
NGAGE generates a Data Warehouse combining a rich profile of each user with a time-stamped record of their every interaction with SharePoint 2013. It makes this data available through a comprehensive set of Microsoft PerformancePoint dashboards and reports that are simple to explore, manipulate and export.
In fact, NGAGE is the only Enterprise Analytics solution built exclusively using Microsoft technologies: it is itself a secure, on-premises SharePoint Service Application - and uses the Microsoft BI stack to deliver a great, self-service analytics experience through SharePoint Business Intelligence Centers.
NGAGEINTELLIGENCE,INC.
Measurably
Successful
SharePoint
Monitor and drive the success of SharePoint
implementation, change management,
communication, training and compliance programs.
Report adoption trends and identify ‘hot spots’ and
‘black holes’ of targeted behaviors at global, departmental, manager and individual level – and by any other available User Property.
Track and improve the quality of the user
experience in terms of search results, navigation, page loading speeds and errors.
Extensible to track adoption of Yammer, HP
WorkSite and virtually any custom or third party application.
To understand and improve how
people and technology are working
together….
…because that’s what determines
the performance of a modern
organization
Enterprise Analytics is for understanding and improving how people and technology are working together. Why is that important?
Because that’s what determines the performance of a modern organization.
Because the value of any investment in technology is only ever a function of how well and how widely it is adopted by its target user population.
And because, while contemporary technologies may bring great benefits – they can also bring reputational, legal and commercial risks.
Within this document you will find information on the scope, granularity and accessibility of the data that NGAGE captures. You’ll find descriptions of its 6 dashboards, 20 universal KPIs and its c.50 standard reports that really are simple to explore, manipulate and export. Also some basic architectural information.
But this is a dynamic application that needs to be seen in action to be fully appreciated.
Because, unlike tools rooted in digital marketing,NGAGE is built for Enterprise Analytics. It provides the big, enterprise-wide picture of how and where technologies are (and are not) being used by
Authenticated Users. And yet you’re never more than a couple of clicks away from the deep insights that allow you to intervene quickly and accurately.
Tracked ‘Contribute’ Behaviors
Site Create, Move, Delete
Document New, Update, Delete, Check In, Check Out
Page New, Update, Delete
Discussion New, Update, Reply, Delete
Asset New, Update, Delete
Blog Post, Comment, Comment Delete
Announcement New, Update, Delete
Contact New, Update, Delete
Link New, Update, Delete
Picture/Media New, Update, Delete
Folder New
List New, Delete
List Field New, Update, Delete
Custom List Item New, Update, Delete
Survey New, Completed
Tracked ‘Connect’ Behaviors
Visited a My Site Updated User Profile
Follow People, Content, Sites, Tags, Unfollow
People Search Like, Unlike Rating - 1-5*
Tracked ‘Consume’ Behaviors
Page ViewDocument Download
Search OnSite, Enterprise
Adoption Measures
User Population Unique Users Adoption Rates First Time Users Repeat Users
Inactive Users Inactive User Rate Behavior Count Behavior Rate Behavior Change %
Rich Behavioral and
Performance Data
NGAGE is a comprehensive Enterprise Analytics solution for SharePoint 2013 On Premises environments.
It harvests a wide range of behavioral and performance data using a number of different techniques and resources – including IIS logs and with Custom Event Receivers tracking changes in SharePoint Lists and Document Libraries.
It also maintains a rich, accurate definition of your Authenticated User Population and your SharePoint infrastructure through its daily User Profile and Farm Topology Synchronization Services.
All this granular data is merged, cleaned and loaded into a SQL Server Data Warehouse and, ultimately, made available through Microsoft PerformancePoint
dashboards.
Sites and Content Reporting
Farm
Web Application Site Collection Site Template Type Site Name Site URL List Content Type File Type Item Name/URL Query String
Other Dimensions and Measures
Hits/Tracked Media Hits Errors/Tracked HTTP Codes Search Terms, Types and Results Search Execution and Elapsed Time Item Loading Speeds
Data Transfer
Click Sources/Destinations Page View Duration Session Duration Daily Usage Patterns
Browser/Operating System/Hardware Content Databases
IP Address
Authenticated User Properties
User ID Display Name Office Location Manager Division Department Job Title Hire Date Email Address Practice Custom Fields Street Address PO Box City State Country Zip Code
Comprehensive, Self-Service Reporting
NGAGE does a good job of collecting granular usage and performance data from SharePoint 2013. But its value is realized in the way this data is surfaced in its out-of-the box dashboards and reports – and how easy these are to explore, manipulate and export.
The credit for that goes primarily to Microsoft PerformancePoint Services. We chose it as our reporting tool because it’s powerful if you want to dig deep and get creative. And because it’s simple to use if you don’t.
And when you want something that PerformancePoint can’t deliver, you can create additional reports using Power View, SQL Server Reporting Services – or any Microsoft SSAS compatible tool.
Reporting Suites
NGAGE ships with two Reporting Suites:
The Master Reporting Suite provides designated ‘super users’ unlimited access to the ‘Single Source of The Truth’ of how SharePoint is being used and experienced by your authenticated users.
The Site Owner Reporting Suite provides access to data only in relation to sites of which the user is the Administrator or a member of its Site Owner Security Group.
Dashboards
Each suite provides a number of dashboards comprising Filters, KPI Scorecards and Reports.
Master Reporting Suite Site Owner Reporting Suite
Behavioral Dashboard
User Experience
Search Analysis
Item Analysis
Data Resource
Filters
All the reports and KPI scorecards within these dashboards can be scoped to any combination of Date, User, Behavior and/or Site using the simple, drop-down Filters which are variously available. Note that, in an improvement over the 2010 edition, Filters in PerformancePoint 2013 are searchable.
Date: Reports/KPIs can be scoped to a specific year (to date), quarter, month or day
Date From/To: Reports can be scoped any date range (Item Analysis Dashboard only)
User by Department: Reports/KPIs can be scoped by the departmental hierarchy retrieved from Active Directory's "Department" field.
User by Manager: Reports/KPIs can be scoped by the organizational hierarchy retrieved from Active Directory's "Manager" field.
Site by Type: Reports/KPIs can be scoped by the custom or standard SharePoint Site Template used to provision the site.
Site: Reports/KPIs can be scoped by SharePoint topology – to Farm, Web Application, Site Collection, Site or Sub-Site levels
Behavior: Reports/KPIs can be scoped to a specific SharePoint feature or behavior – or to another tracked application/feature
KPIs
Both Reporting Suites ship with a set of user adoption and user experience KPI Scorecards. Each KPI within each scorecard will show three values and a ‘traffic light’ indicator of performance.
Actual - This is the value for the KPI appropriate to how you have set the filters.
Target - This is the value for the KPI produced by those same filter settings, except that it is for the previous equivalent time period. i.e. if July is selected in the Date filter, the Target will show the previous month’s (June) value. If 7th May is selected, the Target will show the previous day’s (6th May) value.
Change% - This is the percentage change in the value of the KPI from the Target period to the Actual period.
RED
- The metric has worsened by 10% or more.
AMBER
- The metric is stable within +/- 10%.Behavioral Dashboard
This is essentially the home page within the Master Reporting Suite. It comprises 6 Filters, 4 KPI Scorecards and 20 wholly ‘drillable’ Reports.
Reports
These reports will all be scoped to the selected combination of Date, Site, User and Behavior Filter settings. Note that a ‘Top’ or ‘Bottom’ Report is simply one that has been pre-filtered to present a small set of headline results: it’s simple to modify or clear the filter. Also that the measures and dimensions included in these default reports can removed and/or supplemented with other calculations that are made available through the Select Measures feature.
Contribute
o Additions/Changes to SharePoint Lists and Libraries (Documents, Pages, Discussions, Blogs etc.)
Connect
o Updated My Site User Profile, People Search, Follow, My Site Visits, Likes, Ratings
Consume
o Total Search Count, Page View Count, Document Download Count
Active Users Trend by Behavior Type
o Date, Contribute, Connect, Consume
Site Creations
o Date, Site Template Type
Site Deletions
o Site Type, Deleted Date, Site URL
Top Sites by Behavior
o Active User count, Contribution Count, Adoption Rate, Page View Count
Behavior by Site Type
o Behavior Count, Site Template Type
Bottom Sites by Behavior
o Site is Deleted (Y/N), Deleted Date, Active Users, Page View Count, Document
Download Count, Contribution Count
Top Pages
o Page View Count, Visitor Count, Ave Time on Page (sec), Page View Error Count, Mb Transferred, Media Hit Count
Top Downloads
o Item, Document Download Count, Visitor Count, Avg Document Download Time (sec), Document Download Error Count, Mb Transferred
Top Media Hits
o Media Hit Item, Media Hit Count, Media Hit Error Count, Media Hit Mb Transferred
Top Search Terms
o Search Text, Total Search Count, Average Search Results Count
Top Contributors
o User Name, Department, Behavior Count
Top Visitors
o User Name, Department, Job Title, Page View Count, Document Download Count
Top Proactive Contributions(e.g. new discussion thread, new document, blog post)
o Behavior Type, Behavior Count, Active Users
Top Supportive Contributions (e.g. editing or rating documents, blog comments, discussion replies)
o Behavior Type, Behavior Count, Active Users
Top Item Types
o File Type Group (Web Page, Document, Presentation, Spreadsheet…), File Extension
Top Media Hit Types
o Media Hit Item Type Group (Image…), File Extension
Top Departments
o Department, Visitor Count, Page View Count, Document Download Count, Contributions
User Experience Dashboard
Whatever you’re trying to achieve with SharePoint, you risk falling short if you don’t deliver a consistently high quality user experience. The NGAGE User Experience Dashboard puts you in control with rich reporting and root cause analysis around Errors, Page/Document Downloading speed trends and much more.
With NGAGE, there’s no longer a reliance on anecdotal feedback. For example, it’s simple to compare the real experiences of users across different office locations. With a couple of clicks you can even make sense of the experience of one user on one particular day across any or all sites – right down to item level. The dashboard comprises 5 Filters and 12 Reports.
Reports
Note that ‘Top’,‘Slowest and ‘Fastest’ Reports are simply ones that have been pre-filtered to present a small set of headline results; it’s simple to modify or clear the filter.
Error Trend
o Month, Document Download Error Count, Page View Error Count, Media Hit Error Count
Speed Trend
o Month, Avg Page Download Time, Avg Document Download Time
Search Speed Trend
o Avg Search Execution Time (ms), Avg Search Elapsed Time (ms)
Site Hourly Daily Usage Pattern o Hour, Page View Count
Top Browsers
o Browser Name, Browser Version, Page View Count, Avg Page Load Time (sec), Visitor Count
Top Errors
o Server Response Status Code, Server Response Group (Client/Server Error), Error Description, Total Error Count, Page View Error Count, Document Download Error Count, Media Hit Error Count
Top Error Items
o Error Item URL, Total Error Count
Slowest Pages, Fastest Pages
o Page URL, Avg Page Load Time (sec), Avg Page View Kb Sent, Page View Count
Slowest Downloads, Fastest Downloads
o Item URL, Avg Document Download Time (sec), Avg Document Download Kb Sent, Document Download Count
Peak User Activity
Search Analysis Dashboard
The Search Analysis dashboard provides a KPI Scorecard highlighting performance and adoption trends across all available Search types. The 6 reports provide unique analytics in relation to both On Site and Enterprise Searches – with every metric being drillable down to unique user/search term level.
The quality of the Search experience is pivotal to the success of any ambitious SharePoint implementation. From within the Master Reporting Suite the Search Analysis dashboard provides the ‘big picture’ of how various segments of your organization are engaging with Search.
Meanwhile, Site Owners get to track on site Search terms and activity levels that might suggest a requirement for additional content – or improved navigation to existing content.
Reports
Top Search Terms
o Search Text, Total Search Count, Total User Count, Avg Search Results Count
Rising Search Terms
o Search Text, Total Search Count, Search Count Change %, Total User Count, User Count Change %
Failed Search Terms
o Search Text, Failed Search Count, User Count
Top Search Types
o Search Type, Total Search Count, Search Count Change %, User Count
Search Trend
o Total Search Count, On Site Search Count, Enterprise Search Count, People Search Count, Failed Search Count
Search Speed Trend
Item Analysis Dashboard
This dashboard is for deep analysis of user behavior and experience in relation to specific Pages and Documents over specific time periods. It comprises 3 Filters and 8 Reports. Unlike the other dashboards, these Filters allow for the selection of an individual Item and any date range.
Reports
Usage Trend
o Date, Total Item View Count
Hourly Daily Usage Pattern
o Hour, Page View Count
Click Sources
o Source Site Name, Source Page URL, Total Item View Count, Avg Time on Target Page (sec)
Click Destinations
o Destination Site Name, Destination Page URL, Total Item View Count, Avg Time on Target Page (sec)
Page Performance
o Target Page/Whole Site Comparison: Page View Count, Visitor Count, Avg
Time on Page (sec), Avg Page View kb sent, Avg Page Load Time (sec), Page View Error Count
File Download Performance
o Document Download Count, Visitor Count, Avg Document Download kb sent, Avg Document Download Time (sec), Document Download Error Count
Top Browsers For This Item
o Browser Name, Browser Version, Page View Count, Visitor Count
Top Users
Data Resource Dashboard
This dashboard is unique to the Master Reporting Suite. It is a convenient start point for building bespoke reports and for accessing ‘point data’ very quickly. It provides 5 Filters and 3 Reports
Reports
Behavior Trend
o Month, All Behavior Count
Active Users Trend
o Month, Active User Count
Site Statistics
o Web App/Site Collection/Site Name, Active Users – Consume, Active Users – Contribute, Page View Count, Page View Error Count, Document Download Count, Document Download Error Count, Media Hot Count, Media Hit Error Count, Contribution Count, Total Search Count, Mb Transferred
Site Summary
This dashboard is unique to the Site Owner Reporting Suite. It provides immediate insight into the performance of the selected Site/Site Collection at many levels. It comprises 4 Filters, 3 KPI scorecards and 16 PerformancePoint reports.
Reports
Active Users Trend
o Period, Active User Count
Behavior Trend
o Period, Total Behavior Count
Users by Behavior Type
o Consume/Contribute
Consume
o Page View Count, Document Download Count, Search Count
Contributions by Nature
o Proactive Contribution Count, Supportive Contribution Count
Top Pages
o Page View Count, Visitor Count, Ave Time on Page (sec), Page View Error Count, Mb Transferred, Media Hit Count
Top Downloads
o Item, Document Download Count, Visitor Count, Avg Document Download Time (sec), Document Download Error Count, Mb Transferred
Top Media Hits
o Media Hit Item, Media Hit Count, Media Hit Error Count, Media Hit Mb Transferred
Top Search Terms
o Search Text, Total Search Count, Average Search Results Count
Top Contributors
o User Name, Department, Behavior Count
Top Visitors
o User Name, Department, Job Title, Page View Count, Document Download Count
Top Proactive Contributions(e.g. new discussion thread, new document, blog post)
o Behavior Type, Behavior Count, Active Users
Top Supportive Contributions(e.g. editing or rating documents, blog comments, discussion replies)
o Behavior Type, Behavior Count, Active Users
Top Content Types
o File Type Group (Web Page, Document, Presentation, Spreadsheet…), File Extension
Top Media Hit Types
o Media Hit Item Type Group (Image…), File Extension
Top Departments
o Department, Visitor Count, Page View Count, Document Download Count, Contributions
Architecture
NGAGE for SharePoint 2013 is a Microsoft SharePoint business intelligence solution built for installation in on-premises SharePoint 2013 deployments. It is a SharePoint Farm Solution (WSP), deployed as a Service Application and makes use of the various SharePoint server roles.
The NGAGE Service Application provisioning process creates a custom SQL server database on the SharePoint SQL Server. The Behavior/Meta Data database stores user and organization details, configuration Meta Data and a history of contributions made in SharePoint or other tracked applications.
User information can be imported from Active Directory (plug-in included) or any other user store (simple development, typically <1 day) via a SharePoint Timer Job.
The NGAGE Data warehouse stores all user, organization and behavioral data from the NGAGE database as well as merged and aggregated data from other data sources including IIS logs. SQL Server Integration Services (SSIS) is used to clean and load the data into the data warehouse.
NGAGE Configuration / Management
NGAGE is managed through custom application pages added to SharePoint Central Administration. Both Farm and Web Application scoped features are used to deploy core functionality to the SharePoint Farm. PowerShell is recommended where Farms are large or feature customization is required.
The NGAGE Service Application integrates into the SharePoint Administration framework to facilitate automated backups as part of your standard Farm backup. No additional steps are required to back up the Contribution database or the Service Application.
List Tracking
NGAGE tracks usage of a host of standard SharePoint lists including Announcements, Blogs, Contacts, Custom Lists, Document Libraries, Discussions, Lists, Sites, Surveys and Wikis. Item tracking typically covers Add, Edit/Update, Check-In and Check-Out and is achieved using asynchronous event receivers to ensure there is no impact on either the user experience or server performance.
SharePoint Timer Jobs
A number of timer jobs run nightly to update less frequently changing data such as Users and Farm / Site Topology. These timer jobs can be scheduled to suit the customer’s environment.
PLUGS – Custom and Third Party Application Tracking
Custom and third party web applications can also be tracked by NGAGE using a number of methods including its on-boardREST-based‘PLUGS’functionality. By configuring application specific meta-data and including a call to the NGAGE Logging Web Service, application usage data can be exposed in the reporting dashboards.
Reporting Suites
NGAGE Standard reports are delivered through SharePoint
PerformancePoint Services dashboards. These dashboards are deployed to SharePoint Business Intelligence Center Site Collections and rely on the PerformancePoint Service Application (and Secure Store Application) being provisioned. PerformancePoint is a SharePoint Enterprise Edition feature.
It is important to emphasize the openness of NGAGE in terms of reporting. PerformancePoint is currently the optimal
dashboarding/reporting tool available and it has a number of powerful and unique features, such as Decomposition Tree.
But, as this graphic aims to convey, the NGAGE data warehouse and cube are available for interrogation by any number of other tools that, for some use cases, will be more appropriate.
