May 31, 2012
Abstract Abstract Abstract Abstract
This white paper details global positioning system (GPS) implementation guidance for certification of Windows 8 PCs to help ensure a high-quality and competitive GPS experience in Windows 8. The guidelines in this document apply to OEMs, IHVs, and other partners (such as software vendors). The focus is testing the integration of global navigation satellite system (GNSS) devices to the system under test and Windows 8.
This information applies to the following operating systems: Windows 8 Release Preview
Windows Server 2012
References and resources discussed here are listed at the end of this paper. The current version of this paper is maintained on the Web at:
GNSS (GPS) Test Guidance
Disclaimer Disclaimer Disclaimer
Disclaimer: This document is provided “as-is”. Information and views expressed in this document, including
URL and other Internet website references, may change without notice. Some information relates to pre-released product which may be substantially modified before it’s commercially pre-released. Microsoft makes no warranties, express or implied, with respect to the information provided here. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.
Document Document Document
Document HistoryHistoryHistoryHistory
Date Change
May 31, 2012 First publication
Contents
Introduction... 4
Non-Goals... 4
Test Approach...4
Terminology... 4
Glossary of Terms and Abbreviations... 5
Acceptance Test Matrix...5
Requirements from Partners... 7
Utility to clear the A-GPS data... 7
GPS Acceptance Test Matrix... 7
Telemetry...7
Antenna Requirements from OEM... 7
Operator Certification Tests... 8
Dependency on Third-Party Services or Win32 applications... 8
USB Selective Suspend (if applicable)...8
Firmware Updates... 8
Test Descriptions...8
Functionality... 8
1. Sensor category, type, properties and data fields... 8
2. State Transitions...9
3. Accuracy of Latitude and Longitude...9
4. Minimum Report Interval...10
5. Report Interval Set... 10
6. Report Interval Delay... 10
7. Speed Data... 10
8. Heading Data...11
9. Other Sensor Properties...11
10. Radio State and Flight Mode...11
11. Basic PnP... 11
Assisted GPS...12
1. A-GPS...12
2. Position Injection...12
Robustness...13
1. Driver Verifier, WDF Verifier and Application Verifier...13
2. Telemetry Data...13 3. GPS State Transitions... 14 4. GPS Stress Tests... 14 Performance... 14 1. Cold Start TTFF... 14 2. Hot Start TTFF...15 3. Acquisition Sensitivity... 15
4. Tracking Sensitivity...15
5. Reacquisition Time... 15
6. Static Navigation Accuracy...16
7. Dynamic Navigation Accuracy...16
Power Consumption... 16
1. No GPS Clients...16
2. GPS Radio OFF... 17
3. Radio State and Flight Mode and Power Management...17
4. USB Selective Suspend... 17
Power Management... 17
1. Connected Standby... 17
Antenna Performance...18
1. Performance Tests with OTA Connection... 18
2. Human Interference Tests...19
Interoperability... 20
1. Mobile Broadband, Wi-Fi, Bluetooth, Camera Interoperability...20
Drive Tests... 21
Simulator Tests... 21
Reporting and Results Communication...22
Test Equipment... 22
Toolset... 22
Introduction
This white paper details global positioning system (GPS) implementation guidance for certification of Windows 8 PCs to help ensure a high-quality and competitive GPS experience in Windows 8. The guidelines in this document apply to OEMs, IHVs, and other partners (such as software vendors). The focus is testing the integration of global navigation satellite system (GNSS) devices to the system under test and Windows 8.
Non-Goals
Testing of areas other than GPS is out of the scope of this document. Testing of the components that interact with the location platform and devices will be limited to interoperability testing. Exercising the operating system components or the GNSS device fully is out of scope of this document. It is assumed that the IHVs and OEMs will thoroughly test their GNSS device both independently and integrated into system prior to sending it to Microsoft for managed program certification. This testing should include successful completion of the WHCK tests, this test plan, pre-operator trial tests, and internal tests developed specifically for the GNSS driver and GNSS receiver.
Test Approach
Testing focuses on following areas: 1. Functionality 2. Assisted GPS (A-GPS) 3. Robustness 4. Performance 5. Power 6. Antenna Performance 7. Interoperability 8. Drive Testing 9. Simulator Testing
Managed program partners run the tests specified in this test plan and fill in the
acceptance test matrixbefore submitting the systems/devices/drivers or update to these to Microsoft. The failures in these tests should be documented by the partners and will be evaluated by Microsoft GPS test team whether to accept the device for testing and how to prioritize testing efforts.
Terminology
GPS GPS GPS GPS
GPS is used interchangeably with GNSS. Unless it is stated otherwise, it should be understood to refer generally to a satellite positioning as location provider solution, rather than the Global Positioning System satellite system deployed by the US government.
Clear Clear Clear
Clear skyskyskysky conditionsconditionsconditionsconditions
GPS/GNSS satellites received without obstruction from above or surrounding environment down to an elevation mask of 5 degrees above the horizon. All signal levels consistent with unobstructed signal levels at the ground and not to be lower than -131 dBm.
Glossary of Terms and Abbreviations
GPS: Global Positioning SystemGNSS: Global Navigation Satellite System
GLONASS: Acronym for Globalnaya navigatsionnaya sputnikovaya sistema (in Russian) A-GPS: Assisted GPS, specifically time, coarse position and long-term orbit data TTFF: Time to First Fix
RF: Radio Frequency
WHCK: Windows hardware Certification Kit ADK: Assessment and Deployment Kit DUT: Device under Test
CS: Connected Standby
WDTF: Windows Driver Test Framework SDT: Sensors Diagnostics Tool
SV: Silicon Vendor
IHV: Independent Hardware Vendor OEM: Original Equipment Manufacturer SoC: System on Chip
CTIA: Cellular Telecommunications & Internet Association TIS: Total Isotropic Sensitivity
UHIS: Upper Hemisphere Isotropic Sensitivity PIGS: Partial Isotropic GPS Sensitivity
Acceptance Test Matrix
Acceptance testsAcceptance tests
Test Description Verification
Results Comment s Phase Phase Phase
Phase 1111 (Inbox(Inbox(Inbox(Inbox Criteria)Criteria)Criteria)Criteria)
1 Driver must be signed with the IHV certificate 2 Driver must be installed via device manager/DISM 3 Location Sensor WHCK tests: Device.Input.Sensor.* 4 Radio Management WHCK tests:
System.Client.RadioManagement.*
5 Device Fundamentals WHCK tests: Device.DevFund.* 6 System Fundamentals Power Management WHCK
Tests: System.Fundamentals.PowerManagement.CS.* 7 USB WHCK tests (USB connected devices only):
Device.Connectivity.UsbDevices.*
8 GPS.Test Descriptions.Functionality.Sensor category, type, properties and data fields
9 GPS.Test Descriptions.Functionality.State Transitions 1
0
GPS.Test Descriptions.Functionality.Accuracy of Latitude and Longitude
1 1
GPS.Test Descriptions.Functionality.Minimum Report Interval
1 2
GPS.Test Descriptions.Functionality.Report Interval Set
1 3
GPS.Test Descriptions.Functionality.Speed Data 1
4
GPS.Test Descriptions.Functionality.Heading Data 1
5
GPS.Test Descriptions.Functionality.Radio State and Flight Mode 1 6 GPS.Test Descriptions.Functionality.Basic PnP 1 7
GPS.Test Descriptions.Assisted GPS.A-GPS 1
8
GPS.Test Descriptions.Assisted GPS.Position Injection. Connection Type
1 9
GPS.Test Descriptions. Power Consumption.No GPS Clients
2 0
GPS.Test Descriptions. Power Consumption.GPS Radio Off
2 1
GPS.Test Descriptions. Power
Management.Connected Standby with Active Clients 2
2
GPS.Test Descriptions.Power Management.Resume in No Coverage Area
2 3
GPS.Test Descriptions.Power Management.Connected Standby WHCK Tests with Sensor Active Clients
Acceptance tests 2
4
GPS.Test Descriptions.Antenna Performance.OTA Connection (system should get a fix in clear sky outdoors as is– no external antennas or other modifications)
2 5
GPS.Test Descriptions.Interoperability.* (GPS can get a fix when MB, Bluetooth, Wi-Fi, camera is in active use) Phase
Phase Phase Phase IIIIIIII
1 GPS.Test Descriptions.Functionality.Report Interval Delay
2 GPS.Test Descriptions.Functionality.Other Sensor Properties
3 GPS.Test Descriptions.Assisted GPS.Position Injection. Connection Time
4 GPS.Test Descriptions.Robustness.Driver Verifier, WDF Verifier and Application Verifier
5 GPS.Test Descriptions.Antenna
Performance.HumanInterference Tests (system should get a fix in clear sky outdoors as is– no external antennas or other modifications while being held in common handgrips) Stress Stress Stress Stress 1 GPS.Test Descriptions.Robustness.* Performance Performance Performance Performance 1 GPS.Test Descriptions.Performance.* Power Power Power Power
1 GPS.Test Descriptions.Power Consumption.* 2 GPS.Test Descriptions.Power Management.* 3 System Fundamentals Power Management WHCK
Tests System.Fundamentals.PowerManagement.* Antenna
Antenna Antenna
Antenna PerformancePerformancePerformancePerformance
1 GPS.Test Descriptions.Antenna Performance.* Drive
Drive Drive Drive TestsTestsTestsTests
1 GPS.Test Descriptions.Drive Tests.* Simulator
Simulator Simulator Simulator TestsTestsTestsTests
1 GPS.Test Descriptions.Simulator Tests.*
Requirements from Partners
Utility to clear the A-GPS data
IHVs shall provide a utility to clear the assistance data to enable cold start testing and avoiding false failures in simulator testing. The utility shall have a command line interface.
GPS Acceptance Test Matrix
OEMs and/or IHVs shall run the tests specified in the GPS Acceptance Test Matrix before submitting the system/device/driver to test and document the test results in this matrix.
Telemetry
Prior to submitting updated drivers, IHVs should review OCA and Watson crashes caused by their GPS drivers and fix all frequent crashes.
Antenna Requirements from OEM
1. Managed systems with GPS support shall be tested according to the Cellular Telecommunications & Internet Association (CTIA) CTIA Test Plan for Mobile Station Over-the-Air Performance, Method of Measurement for Radiated RF Power and Receiver Performance v3.0+ for A-GPS and shall pass it. Total Isotropic Sensitivity (TIS), Upper Hemisphere Isotropic Sensitivity (UHIS) and Partial
Isotropic GPS Sensitivity (PIGS) will be measured and OEMs shall post measurement results to Microsoft for review.
2. The system must have freespace TIS and UHIS of -140 dBm or better for GPS. The measurements must follow the test methodology and test parameters defined in CTIA 3.x test plan.
3. The average gain of the GPS antenna should be better than -6dBi.
4. Human hands holding the device in one of the common handgrips should not cause performance to drop below the minimal acceptable standard. Device should be able to maintain OTA acquisition sensitivity at -140 dBm and OTA tracking sensitivity of -145 dBm when the system is held in common positions. 5. Antenna performance and radiated sensitivity testing should be performed when
the GPS antenna is in their expected positions within the device.
6. EV systems should have the intended GPS antenna for production added in its intended location for production and DV systems should have antenna location finalized.
7. OEMs should run antenna performance and radiated sensitivity tests and
understand the failures prior to sending EV units. The tests must pass prior to DV units.
Operator Certification Tests
If operator certification tests for A-GPS are executed, pass/fail results and waiver information is to be shared with Microsoft.
Dependency on Third-Party Services or Win32 applications
There shall be no dependency on third-party services or Win32 applications accompanying the GPS solution. Dependency on third-party services has power implications, hence not allowed. Third-party Win32 applications are subject to signing requirements on SoC systems and not allowed.USB Selective Suspend (if applicable)
USB connected GPS devices shall support selective suspend.
Firmware Updates
GPS on mobile broadband modules shall update via UEFI and standalone GPS shall update via driver.
Test Descriptions
Functionality
WHCK tests that apply to GNSS devices are the first set of tests to verify the basic functionality of GPS devices. WHCK contains tests for GPS Sensors, Radio Manager, Device Fundamentals, System Fundamentals Power Management, Windows Driver Test Framework (WDTF), and USB Hardware Certification tests (for USB connected devices) that apply to GNSS devices.
More information about WHCK tests can be found athttp://connect.microsoft.com/. The functionality tests are intended to run under clear sky conditions.
1. Sensor category, type, properties and data fields
Description: Description: Description:
Description: The device should report correct sensor category and type and support
mandatory properties and data fields. In addition to the mandatory sensor properties documented in MSDN, speed data in knots (SENSOR_DATA_TYPE_SPEED_KNOTS) and heading data relative to true north (SENSOR_DATA_TYPE_TRUE_HEADING_DEGREES) are mandatory for managed program systems.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Query sensor category, type, properties and data fields the Device
Under Test (DUT) reports. Sensor Diagnostics Tool (SDT) in Windows Driver Kit (WDK) can be used to test this.
Expected Expected Expected
Expected Result:Result:Result:Result: Mandatory fields must be supported and report correct data.
2. State Transitions
Description: Description: Description:
Description: The device should report changes in sensor state, as documented in
MSDN.
• Data reports should only be reported after device reaches to SENSOR_STATE_READY or SENSOR_STATE_INITIALIZING.
• Device should not report data if it does not have lat/long information, e.g. time only.
• GPS sensor should start in SENSOR_STATE_INITIALIZING.
• GPS sensor should continue to acquire fix and stay in initializing state until the request is cancelled by the OS.
• GPS Sensor should go into SENSOR_STATE_READY when it gets a fix and has data.
• GPS Sensor should go into SENSOR_STATE_INITIALIZING when it loses the signal and no longer has data. It should go back to SENSOR_STATE_READY when it reacquires fix.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Disable and re-enable the device. Monitor state transitions and data
events. Move into an area with no GPS signal (e.g. faraday cage), wait for at least one minute, and move back into coverage area. Monitor state transitions and data events.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should report sensor state transitions e.g. from ‘initializing’
to ‘ready’. Data reports should only be reported after reaching to ‘initializing’ and ‘ready’ states. Data should not be reported if latitude and longitude information is not available. Device should start in ‘initializing’ state and should not move into ‘ready’ state before getting a fix and has a valid error radius. When moved out of GPS signal coverage area, the device should go into SENSOR_STATE_INITIALIZING and it should go into SENSOR_STATE_READY when moved back to coverage area.
3. Accuracy of Latitude and Longitude
Description: Description: Description:
Description: The device should provide accurate latitude and longitude values within
the specified error radius.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: During static testing and in vehicle testing, the data from device
under test (DUT) will be compared to latitude and longitude data reported by reference GPS, survey markers and simulator.
Expected Expected Expected
Expected Result:Result:Result:Result: The difference between the latitude and longitude values reported
by DUT and the reference GPS should be within the error radius DUT reported.
4. Minimum Report Interval
Description: Description: Description:
Description: GPS driver should not send data reports more frequent than current
report interval.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Query the current report interval. Monitor the time of the previous
and latest data reports. Compare the time between data reports to current report interval.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should not send data reports more often than current report
interval.
5. Report Interval Set
Description: Description: Description:
Description: GPS driver shall support setting the report interval. Device should be
able to report data at 1Hz frequency. It should calculate the minimum report interval in case of multiple clients setting the report interval to different values. It should reflect changes from clients going away and new clients coming in and setting report interval. Driver should follow the guidelines in MSDN managing multiple clients setting report interval.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Set report interval from multiple clients. Query the current report
interval. Monitor the time of the previous and latest data reports. Compare the time between data reports to current report interval. Add new clients changing the report interval, remove clients who previously set the report interval.
Report intervals to set: • 1 seconds • 15 seconds • 1 minute Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to report data at 1Hz frequency. Device
should support setting report interval and should correctly reflect changes in the report interval from multiple clients. It should keep report interval up to date with new clients coming in and going away. It should not send data reports more often than current report interval.
6. Report Interval Delay
Description: Description: Description:
Description: GPS driver should not delay data reports more than 100% of the
currently set report interval when it is in ‘ready’ state.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Query the current report interval. Monitor the time of the previous
and latest data reports. Compare the time between data reports to current report interval.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should not delay data reports more than 100% of the
currently set report interval.
7. Speed Data
Description: Description: Description:
Description: DUT should report speed data in knots when moving. Execution
Execution Execution
Execution Steps:Steps:Steps:Steps: Perform simulated in vehicle tests, manual walking tests and drive
tests and monitor the speed data reported by device.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should report speed data, and accuracy of the speed
information should be within ±10% of the speed data reported by reference GPS or simulator.
8. Heading Data
Description: Description: Description:
Description: DUT should report heading in degrees relative to true north when
moving.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Perform simulated in vehicle tests, manual walking tests and drive
tests and monitor the heading data reported by device.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should report heading data, and accuracy of the heading
degrees should be within ±10% of the speed data reported by reference GPS or simulator.
9. Other Sensor Properties
Description: Description: Description:
Description: If other sensor properties are supported, they should report valid data
and accurate values.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Monitor properties supported by DUT and ensure they provide valid
data within acceptable accuracy.
Expected Expected Expected
Expected Result:Result:Result:Result: If a sensor property is supported, they should report accurate
values within ±20% of the reference GPS or simulator reported values.
10. Radio State and Flight Mode
Description: Description: Description:
Description: Turning flight mode on or turning GPS radio off from Windows Control
Panel should turn off the GPS radio. Device should go into lowest possible power state when radio is off. Device should be able to acquire fix after turning the radio back on.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: While a location client application is active (e.g. sensor diagnostic
tool), turn flight mode on or GPS radio off. Monitor data and state change events from DUT. Turn radio back on or turn off flight mode. Try to acquire fix.
Expected Expected Expected
Expected Result:Result:Result:Result: Driver should stop reporting data events and change sensor state to
SENSOR_STATE_NOT_AVAILABLE. Device should go into lowest possible power state. After turning the radio on or turning the flight mode off, it should be able to acquire fix.
11. Basic PnP
Description: Description: Description:
Description: Uninstalling and re-installing the GPS driver, disabling and re-enabling
the device should work as expected.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: While a location client application is active (e.g. sensor diagnostic
tool), disable and re-enable the device. Attempt to acquire fix. When device is in use again, uninstall the driver and then reinstall.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to get disabled and re-enabled. It should be
able to acquire fix after enable. Driver should be able to get uninstalled gracefully.
Assisted GPS
When the GPS is first powered up, it should be able to use “assisted GPS” to return an approximated location in a few seconds. When using assisted GPS, the sensor should provide location data, which can be several hundred meters to six figure numbers. Later as the GPS radio is able to obtain multiple satellite locks, the error radius should decrease to a value between 3 – 30 meters.
1. A-GPS
Description: Description: Description:
Description: A-GPS should help getting a faster TTF, with lower accuracy. Execution
Execution Execution
Execution Steps:Steps:Steps:Steps: Cold start the GPS device. Monitor latitude, longitude and error
radius data fields from Sensor Diagnostics Tool (SDT). Based on the following conditions:
• Clear-sky conditions (simulated or actual) • Subscribed for data events
• Report interval of one second
• Wi-Fi or cellular baseband is present and enabled
Expected Expected Expected
Expected Result:Result:Result:Result: Device should return a position from A-GPS as soon as possible and
associated error radius should be reported. The higher error radius (e.g. 300 meters if Wi-Fi is available) should decrease to 3-30 meters as device acquires multiple satellite locks. GPS should report a position based on assistance data within 15 seconds.
2. Position Injection
GPS driver can use data from triangulation sensors present on the system to speed up Time To First Fix (TTFF) using Sensor API.
a. a. a.
a. ConnectionConnectionConnectionConnection TimeTimeTimeTime
Description: Description: Description:
Description: GPS driver should close the connection to other sensors immediately
after getting the position. It should timeout after 15 seconds and closes the connection to Sensor API if it does not get a position.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Start monitoring the traces from Sensor API for the active client
counts for all sensors in the system. Cold start the GPS device. Monitor the changes on the active client counts for other sensors in the system.
Expected Expected Expected
Expected Result:Result:Result:Result: If active client counts for other sensors are incremented, they
should return to their previously recorded values after the 15 seconds. b.
b. b.
b. ConnectionConnectionConnectionConnection TypeTypeTypeType
Description: Description: Description:
Description: GPS drivers should not instantiate ILocation to get data from other
location sensors. They can use the Sensor API (ISensorManager) to open connection to for instance triangulation sensors
(SENSOR_TYPE_LOCATION_TRIANGULATION). GPS driver should not get data from location sensors of the same type. For instance, a GPS sensor should not use data from other sensors with type GPS to get a faster fix.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Find out the sensor type the DUT is reporting e.g.
SENSOR_TYPE_LOCATION_GPS. Disable all sensors except the sensors of the same type with the DUT. Start monitoring the traces from Sensor API for the active client counts for enabled sensors in the system. Cold start the GPS device. Monitor the changes on the active client counts for sensors in the system.
Expected Expected Expected
Expected Result:Result:Result:Result: Active client counts for the sensors of the same type should not
be incremented.
Robustness
Driver Verifier, WDF Verifier and Application Verifier will be enabled for location platform and GPS device stack during all testing to test the reliability of the GPS support in the system.
1. Driver Verifier, WDF Verifier and Application Verifier
Description Description Description
Description: Enable Application Verifier and Driver Verifier at the beginning of testing. Execution
Execution Execution
Execution StepsStepsStepsSteps: Enable Driver Verifier on all kernel mode drivers in the driver
package (if any) and Application Verifier for %windir%\system32\WUDFHost.exe for WDF drivers at the beginning of testing. Enable Driver Verifier on WDF drivers and any kernel mode drivers you have.
Application Verifier (appverif.exe) is available in WHCK Client and SDK. A minimum of basic settings without ‘Leak’ detection is required.
Driver Verifier is part of the OS builds. It can be launched from admin command prompt with following settings for kernel mode drivers:
Verifier /standard /driver wudfpf.sys Wdf01000.sys Wdfldr.sys wudfrd.sys <any kernel mode driver you have>
WDFVerifier is enabled by default for all WDF drivers. “WdfVerifier.exe’ tool in WDK can be utilized to control the verbosity of logging, debugger settings etc.
Expected Expected Expected
Expected ResultResultResultResult: No verifier crashes.
2. Telemetry Data
Description Description Description
Description: Monitor telemetry data from Microsoft device web for the GPS driver. Execution
Execution Execution
Execution StepsStepsStepsSteps: Monitor telemetry data from Microsoft device web for the GPS
driver. Identify, investigate and fix the crashes in the driver.
Expected Expected Expected
Expected ResultResultResultResult: All telemetry reported crashed should be triaged and the top
crashes should be investigated and fixed.
3. GPS State Transitions
Description Description Description
Description: System will be cycled through 5 transitions of entering and exiting
environment where a GPS signal can and cannot be received. GPS sensor state transitions will be verified among the ready and initializing states as expected.
Execution Execution Execution
Execution StepsStepsStepsSteps: 5 transitions of entering and exiting environment where a GPS signal
can and cannot be received.
Expected Expected Expected
Expected ResultResultResultResult: GPS state should change to/from ready and initializing as expected.
Device should be able to get a fix after these iterations.
4. GPS Stress Tests
A combination of the operations listed below will be performed simultaneously on the GPS device during simulator scenario execution, walk testing and drive testing:
• Enable Driver Verifier • Enable Application Verifier
• Repeated WHCK test run (GPS Sensor, Radio Manager, System Power Management)
• Manual operations on Sensor Diagnostics Tool • Manual Radio Management Operations • Manual Connected Standby
• Manual GPS device disable/re-enable.
• Windows Location Provider disable/re-enable • Mobile Broadband or Wi-Fi device disable/re-enable • Large download over Mobile Broadband connection • Large download over Wi-Fi connection
• Bluetooth activity
A basic verification test will be performed before the stress testing. The expectation is that the same basic verification test to pass after the stress and no crashes are
observed during testing.
Performance
GPS device performance will be tested for cold start TTFF, hot start TTFF, acquisition sensitivity, tracking sensitivity, re-acquisition time, static navigation accuracy, and dynamic navigation accuracy.
GNNS Simulator will be utilized for performance testing.
1. Cold Start TTFF
Description: Description: Description:
Description: Cold Start TTFF should be achieved in less than 45 seconds 90% of the
time. Cold start is described as: • Time unknown
• Current ephemeris unknown • Position unknown
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Clear the GPS assistance data before the test. Ensure the cold start
conditions described above. Monitor the TTFF under clear sky conditions (actual or simulated).
Expected Expected Expected
Expected Result:Result:Result:Result: Device should acquire a fix using the GNSS device within 45 seconds
90% of the time.
2. Hot Start TTFF
Description: Description: Description:
Description: Hot Start TTFF should be achieved within 2 seconds. Hot start is
described as:
• Time is known • Almanac is known • Ephemeris is known
• Position within 100 km of last fix
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Ensure the hot start conditions described above. Monitor the TTFF
under simulated clear sky conditions.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should acquire a fix using the GNSS device within 2 seconds
90% of the time.
3. Acquisition Sensitivity
Description: Description: Description:
Description: Device should be able to obtain a fix at -150 dBm or lower power levels. Execution
Execution Execution
Execution Steps:Steps:Steps:Steps: Under simulated lab conditions, with a direct RF connection when
antenna connector is accessible, expose device to low power levels up to -150 dBm.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should acquire a fix at -150 dBm.
4. Tracking Sensitivity
Description: Description: Description:
Description: Device should be able to maintain a fix at -155 dBm or lower power
levels.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Under simulated lab conditions, with a direct RF connection when
antenna connector is accessible, after obtaining a fix, reduce the power level down to -155 dBm.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to maintain a fix -155 dBm.
5. Reacquisition Time
Description: Description: Description:
Description: Device should be able to reacquire fix within 2 seconds. Clear sky
conditions assumed when signal is available.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Under simulated lab conditions, after obtaining a fix, reduce the
power level down enough to get the DUT lose the fix. Then increase the power levels back and monitor the reacquisition time. Alternatively, drive through a tunnel during drive testing.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to reacquire fix within 2 seconds.
6. Static Navigation Accuracy
Description: Description: Description:
Description: Device should be able to report accurate latitude, longitude and if
supported, altitude.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Under simulated ideal lab conditions, compare the accuracy of
longitude, latitude and if available, altitude to the simulated location.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to report horizontal accuracy of 15 meters
and vertical accuracy of 30 meters at 95% of the time.
7. Dynamic Navigation Accuracy
Description: Description: Description:
Description: Device should be able to report accurate latitude, longitude and if
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Under simulated ideal lab conditions, simulate in vehicle motion and
compare the accuracy of longitude, latitude and if available, altitude to the simulated values.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should be able to report horizontal accuracy of 15 meters
and vertical accuracy of 100 meters.
Power Consumption
Battery life run down testing should be performed by evaluating battery life while executing predefined workloads described in ADK. The following are the scenarios that will be used to evaluate system power performance for GPS.
• GPS in use with clients subscribed at default report interval, high accuracy of data requested.
• Hourly lookups
• Location platform enabled. All previously connected clients disconnected. • Location platform enabled. Flight mode on.
• Location platform disabled.
1. No GPS Clients
Description: Description: Description:
Description: When there are no GPS clients connected to the location platform or
subscribed for data update events, GPS device should go into D3, and be in the lowest possible power state. When there is a connected client but it’s not subscribed for data update events (single shot or polling), GPS should acquire a fix and then enter low power tracking mode until client asks for a position.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Close all client applications (e.g. SDT). Monitor device power state
from device manager (Device Properties/Details/Power Data). Launch a client app that only performs single shot look ups and does not subscribe for data events. Perform single shot look ups between long time intervals while moving and monitor the time to fix and accuracy of the data.
Expected Expected Expected
Expected Result:Result:Result:Result: GPS device should go into D3 when there are no connected clients.
Device should go into low power tracking mode after single shot look ups.
2. GPS Radio OFF
When GPS radio is turned off, GPS device should go into the lowest power state possible but not remove itself from the bus.
3. Radio State and Flight Mode and Power Management
Description: Description: Description:
Description: The GPS driver needs to incorporate the radio state in power
management handling. Device should go into lowest possible power state when radio is off. When active client count goes to 0 when radio is off, device should go into D3. When a client connects when radio is on, Device should re-enter D0, however, should only fully power on when the radio is enabled.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: While a location client application is active (e.g. SDT), turn flight
mode on or GPS radio off. Monitor data and state change events from DUT. Close client application (e.g. SDT). Monitor device power state (e.g. from Device Manager: Device Properties/Details/Power Data). Launch the client application again. Monitor device power state. Turn radio back on or turn off flight mode. Try to acquire fix.
Expected Expected Expected
Expected Result:Result:Result:Result: When radio is turned off, device should stop reporting data events
and change sensor state to SENSOR_STATE_NOT_AVAILABLE. Device should go into lowest possible power state. After closing the client application, device should go into D3. When client application is re-launched, device should go back to D0, however should not fully power on. After turning the radio on or turning the flight mode off, it should be able to acquire fix.
4. USB Selective Suspend
This test applies to USB connected devices only. GPS device with no clients subscribed for a report interval of 8 seconds or less should participate on selective suspend when all devices on the bus is ready to go into suspend mode.
Device Manager and ETW events will be used to monitor the USB bus state transitions.
Power Management
1. Connected Standby
New Driver Verifier tools for Runtime Power and Location Platform Plug in for Windows Driver Test Framework (WDTF) will be utilized for Connected Standby (CS) testing. Driver Verifier Run Time Power tests will be enabled throughout the certification testing with the exclusion of power consumption and performance tests.
a. a. a.
a. ConnectedConnectedConnected StandbyConnectedStandbyStandbyStandby withwith ActivewithwithActiveActiveActive ClientsClientsClientsClients
Description: Description: Description:
Description: Put system into Connected Standby state when there are active
clients and device is in ‘ready’ state. Device should go into Connected
Standby. Resume in coverage area after 30 seconds. Device should be able to acquire fix and report data within 2 seconds after resuming.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: When there are active clients connected put the machine
into connected standby. Wake from connected standby.
Expected Expected Expected
Expected Result:Result:Result:Result: Device should go into Connected Standby mode. After
resume, it should reacquire fix and start reporting data within 2 seconds. b.
b. b.
b. ResumeResumeResumeResume inininin nononono CoverageCoverageCoverageCoverage AreaAreaAreaArea
Description: Description: Description:
Description: Put system into Connected Standby state when there are active
clients. Resume in no coverage area. Device should try to acquire fix and enter ‘initializing’ state.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: When there are active clients connected put the machine
into connected standby. Wake from connected standby in an area without GPS signal.
Expected Expected Expected
c. c. c.
c. ConnectedConnectedConnectedConnected StandbyStandbyStandbyStandby WHCKWHCKWHCKWHCK TestsTests withTestsTestswithwithwith SensorSensorSensorSensor ActiveActiveActiveActive ClientsClientsClientsClients
Description: Description: Description:
Description: Start a GPS client application which is subscribed for data events
and run System Fundamentals Power Management WHCK Tests.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Start a sensor client application which is subscribed for data
events and run System Fundamentals Power Management WHCK Tests: System.Fundamentals.PowerManagement.CS.*
Expected Expected Expected
Expected Result:Result:Result:Result: Tests should pass and GPS device should get a fix and
continue to function as expected at the end of the test.
Antenna Performance
1. Performance Tests with OTA Connection
It is common to do performance testing of GNSS receivers in lab environment over a cabled RF connection, bypassing the GPS antenna and associated circuitry. This approach does not give a complete picture of real-world device performance and issues in GPS antenna and its circuitry can cause poor user experience in location based service applications. To catch these issues, managed system device should be tested for GPS performance with an over-the-air (OTA) test methodology.
Managed systems with GPS support shall be tested according to the Cellular
Telecommunications & Internet Association (CTIA) CTIA Test Plan for Mobile Station Over-the-Air Performance, Method of Measurement for Radiated RF Power and Receiver Performance v3.0+ for A-GPS and shall pass it. Total Isotropic Sensitivity (TIS), Upper Hemisphere Isotropic Sensitivity (UHIS) and Partial Isotropic GPS Sensitivity (PIGS) will be measured. OEMs shall share results with Microsoft and let Microsoft know of any waiver scenarios.
The system must have freespace TIS and UHIS of -140 dBm or better for GPS. The measurements must follow the test methodology and test parameters defined in CTIA 3.x test plan.
The average gain of the GPS antenna should be better than -6dBi.
Human hands holding the device in one of the common handgrips should not cause performance to drop below the minimal acceptable standard. Device should be able to maintain OTA acquisition sensitivity at 140 dBm and OTA tracking sensitivity of -145 dBm when the system is held in common positions.
In addition to antenna design and human interference from holding the device, device form factor and location of the GPS antenna also impact the antenna pattern. Therefore, antenna performance and radiated sensitivity testing should be performed when the GPS antenna is in their expected positions within the device.
2. Human Interference Tests
Antenna positioning should take human interference into account. When the system is held in common states, GPS should not lose fix, should not increase the error radius
more than 30%, and should be able to maintain OTA acquisition sensitivity at -140 dBm and OTA tracking sensitivity of -145 dBm.
Commonly used states for slates:
• Hands on the sides. Landscape orientation • Hand on bottom, Landscape Orientation
• Hands on the sides. Portrait orientation (Start on Left) • Hands on the bottom. Portrait orientation (Start on Left) • Hands on the sides. Portrait orientation (Start on Right) • Hands on the bottom. Portrait orientation (Start on Right)
a. a. a.
a. HumanHumanHuman InterferenceHumanInterferenceInterferenceInterference ImpactImpactImpactImpact onononon SensorSensor StateSensorSensorStateStateState andandandand ErrorErrorErrorError RadiusRadiusRadiusRadius
Description: Description: Description:
Description: The sensor state and error radius should not cause performance
to drop below the minimal acceptable standard when it’s hold at specified handgrips.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Under clear sky conditions, when device is in ready state,
hold the device at commonly used states. Check sensor state and error radius.
Expected Expected Expected
Expected Result:Result:Result:Result: Device state and error should not cause performance to
drop below the minimal acceptable standard. Device should not lose fix and increase the error radius more than 30%.
b. b. b.
b. HumanHumanHumanHuman InterferenceInterferenceInterferenceInterference ImpactImpactImpactImpact onon AcquisitionononAcquisitionAcquisitionAcquisition andandandand TrackingTrackingTrackingTracking SensitivitySensitivitySensitivitySensitivity
Description: Description: Description:
Description: The device acquisition and tracking sensitivity should not cause
performance to drop below the minimal acceptable standard when it’s hold at specified handgrips.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Hold the device at commonly used states. Check acquisition
sensitivity and tracking sensitivity.
Expected Expected Expected
Expected Result:Result:Result:Result: Tracking and acquisition sensitivity should not be impacted
when the device is held on specific locations. Device should be able to maintain OTA acquisition sensitivity at -140 dBm and OTA tracking sensitivity of -145 dBm.
Interoperability
1. Mobile Broadband, Wi-Fi, Bluetooth, Camera Interoperability
Other radios and devices such as camera in the system can cause interference with GPS. It is also common that GPS device share the same module with mobile
broadband, Wi-Fi and Bluetooth. GPS functionality should not be impacted by usage of these devices.
Description: Description: Description:
Description: Simultaneous usage of Mobile broadband, Wi-Fi, Bluetooth and Camera
should not degrade the performance and functionality of the GPS device. Active usage of GPS should not impact the functionality of these devices.
Execution Execution Execution
Execution Steps:Steps:Steps:Steps: Run basic functionality tests with MB, Wi-Fi, Bluetooth and Camera
on and actively in use.
• Large download over Mobile Broadband connection when using GPS. Monitor SDT and the event logs from SDT.
• Large download over Wi-Fi connection when using GPS. Monitor SDT and the event logs from SDT.
• Bluetooth file transfer while using GPS. Monitor SDT and the event logs from SDT.
• Wi-Fi scan while using GPS. Monitor SDT and the event logs from SDT. • MB scan while using GPS. Monitor SDT and the event logs from SDT. • BT scan while using GPS. Monitor SDT and the event logs from SDT.
• Video recording while using GPS. Monitor SDT and the event logs from SDT. • Watch movie over internet (e.g. Netflix) while using GPS. Monitor SDT and
the event logs from SDT.
Expected Expected Expected
Expected Result:Result:Result:Result: Simultaneous usage of these devices should not impact GPS and
these devices functionality. GPS should continue to function when these devices in use. If these devices are functioning well without GPS being in use, they should continue to function well when GPS is in use, too.
Drive Tests
Manual drive testing where the system is taken on a drive on a route defined earlier which includes tunnels and areas with different multipath impact. During the drive, the system GPS data will be captured by a test application and will be compared to a reference GPS. At no time should the location reported by the system +/- the error radius be outside the location reported by the reference GPS. In addition, the average error radius should be <= 30 meters....
Drive testing will exercise real life conditions i.e. dynamic navigation accuracy, reacquisition after going through a tunnel, impact of multi path and atmospheric conditions.
Following functionality tests will be run during drive testing:
• State transitions will be monitored and compared to reference GPS.
• Latitude, Longitude and if available altitude will be monitored and compared to reference GPS.
• Reacquisition time after going through the tunnels will be measured and compared to reference GPS.
• Tracking sensitivity will be monitored in multipath impact areas with regards to reference GPS.
• Dynamic navigation accuracy will be monitored and compared to reference GPS.
• Device PnP and radio manager state transitions will be performed during dynamic navigation.
Simulator Tests
A GNSS simulator (Spirent GSS6700) will be utilized to achieve controlled lab
conditions, replaying same test scenarios for repeatability, simulating satellite states, various locations and time e.g. south of equator and 2 years later, simulate in vehicle navigation, atmospheric conditions, multi-path and error conditions.
Over The Air RF connection will be used to be able to test the system with original antenna and shielding. Direct connection may also be used when antenna connector is accessible for receiver testing. Simulator testing will focus on the common
simulator scenarios including GNSS receiver performance characteristics and special scenarios: • COLD Start TTFF • HOT Start TTFF • Acquisition Sensitivity • Re-acquisition Sensitivity • Tracking Sensitivity • Static Position Accuracy • Dynamic Position Accuracy • Dynamic Position Accuracy • Multipath
• GPS and GLONASS
The standard scenarios supplied with the Spirent GSS6700 simulator will be run.
Reporting and Results Communication
All issues found will be communicated to partners via bugs.
The feedback on the test categories listed above and their sub-categories will be provided in the form of a color coded score card. Each test area will be represented with a scorecard and the goal is having all scenarios green.
We will also provide the test logs e.g. WHCK logs, traces, logs from partner driver and any relevant performance results and baseline performance comparison data.
Test Equipment
Following test equipment will be used: • Spirent GSS6700 GNSS Simulator • Faraday Cage • RF shielding box • MB SIM • Reference Devices • External antennas
Toolset
WHCK Driver Verifier WDF Verifier Application VerifierSensor Diagnostics Tool: WDKToolRoot\<architecture>\sensordiagnostictool.exe
Resources
Mandatory Mandatory Mandatory
Mandatory sensorsensorsensorsensor propertiespropertiespropertiesproperties documenteddocumenteddocumenteddocumented inininin MSDN:MSDN:MSDN:MSDN:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff545919(v=vs.85).aspx
Guidelines Guidelines Guidelines
Guidelines inininin MSDNMSDNMSDNMSDN managingmanagingmanagingmanaging multiplemultiplemultiple clientsmultipleclientsclientsclients settingsettingsettingsetting reportreportreportreport interval:interval:interval:interval:
http://msdn.microsoft.com/en-us/library/windows/hardware/Hh706203(v=vs.85).aspx http://msdn.microsoft.com/en-us/library/windows/hardware/hh706201(v=vs.85).aspx. How How How
How totototo turnturnturnturn WDFWDFWDFWDF VerifierVerifierVerifierVerifier on:on:on:on:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff556127(v=vs.85).aspx
other other other
other usefulusefulusefuluseful infoinfoinfoinfo onononon WDFWDFWDFWDF Verifier:Verifier:Verifier:Verifier:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff556129(v=vs.85).aspx
Application Application Application
Application Verifier:Verifier:Verifier:Verifier:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff538115(v=vs.85).aspx
Driver Driver Driver
Driver Verifier:Verifier:Verifier:Verifier:
