Section 9 Recommendations
9.5 Knowledge Base Gaps
A number of actions that could help bridge the gaps listed in the previous three sections are focused on increasing the knowledge base of CS/IA measurement practitioners by—
1. Leveraging measurement expertise and lessons learned from other industries—Common expectations, challenges, and success factors, articulated in Sections 9.1 and 9.2, are not unique to the CS/IA industry. While many challenges are specific to CS/IA, many are of an organizational nature. Measurement experts in other industries have successfully managed and overcome many of these challenges.
Knowledge that exists within other industries can increase the cost-effectiveness and success rate of CS/IA measurement efforts.
2. Creating a skilled/trained labor force dedicated to CS/IA measures—
Building the CS/IA measurement knowledge base and resource pool is critical to the success of CS/IA measurement efforts. The current workforce is knowledgeable about IA or about measurement, but it is rare that both skill sets are present. Investing in training IA
practitioners in measurement methods and techniques would increase cost-effectiveness and success ratio of current and future CS/IA measurement efforts.
References
170 Victor-Valeriu Patriciu, Iustin Priescu, and Sebastian Nicolaescu. “Security Metrics for Enterprise Information Systems,” in Journal of Applied Quantitative Methods (JAQM), Vol. 1, No. 2, Winter 2006.
Accessed 6 January 2009 at: http://jaqm.ro/issues/volume-1,issue-2/pdfs/patriciu_priescu_nicolaescu.pdf 171 Chew, et al. Performance Measurement Guide for Information Security, op cit.
172 Michael S. Mimoso, Editor. “Number-driven risk metrics ‘fundamentally broken’,”
in Information Security, 12 March 2009. Accessed March 26, 2009 at:
http://searchsecurity.techtarget.com/news/article/0,289142,sid14_gci1350658,00.html#
173 Ibid.
A Abbreviations, Acronyms, and
Definitions
Acronym Definition
ACSAC Annual Computer Security Applications Conference ActSec Actual Security
AES Advanced Encryption Standard AFCA Air Force Communication Agency AFIT Air Force Information Technology
AG Attack Group
ALE Annualized Loss Expectancy
AMBER Assessing, Measuring, and Benchmarking Resilience
AmI Ambient Intelligence
ANSI American National Standards Institute ARO Annualized Rate of Occurrence
ARO Army Research Office
ARR Attack Relevance Rating AS&W Attack Sensing and Warning
ASVS Application Security Verification Standard AT/SPI Anti-Tamper/Software Protection Initiative ATO Authorization to Operate
BAR Business Adjusted Risk BJS Bureau of Justice Statistics BOF Birds of a Feather
BRM Business Reference Model C&A Certification and Accreditation
CAVP Cryptographic Algorithm Validation Program
CC Common Criteria
CCE Common Configurations Enumeration CCSS Common Configurations Scoring System CHACS Center for High Assurance Computer Systems CI/KR Critical Infrastructure and Key Resources CCTL Common Criteria Testing Laboratories CIO Chief Information Officer
CIP Critical Infrastructure Protection CIS Center for Internet Security CISO Chief Information Security Officer
CISWG Corporate Information Security Working Group CJCSI Chairman of the Joint Chiefs of Staff Instruction CMM Capability Maturity Models
CMMI Capability Maturity Model Integration CMSS Common Misuse Scoring System CMU Carnegie Mellon University
CMVP Cryptographic Module Validation Program CND Computer Network Defense
CNDSP Computer Network Defense Service Provider CNO Computer Network Operations
CNRS-LAAS Université de Toulouse Centre Nationale de la Recherche Scientifique Laboratoire d’Analyse et d’Architecture Systemès
COTS Commercial Off the Shelf
CPE Common Platform Enumeration
CR/TA Critical Review/Technology Assessment CS/IA Cyber Security and Information Assurance CSIS Center for Secure Information Systems CSO Chief Security Officer
CSR Critical Security Rating CSS Computer Security Survey
CVE Common Vulnerabilities and Exposures
CVSS Common Vulnerabilities Common Scoring System
CWE Common Weakness Enumeration
CWSS Common Weakness Scoring System
DARPA Defense Advanced Research Projects Agency
DEPEND Design and Validation of Reliable Networked Systems DESEREC Dependability and Security by Enhanced ReConfigurability
DHS Department of Homeland
DIACAP Defense Information Assurance Certification and
DIAP Defense-wide Information Assurance Program D-IART Defense-Information Assurance Red Team DISA Defense Information Systems Agency DITSCAP Defense Technology Security Certification
and Accreditation DLA Defense Logistics Agency
DoD Department of Defense
DON Department of Navy
DON CIO Department of Navy Chief Information Officer DREAD Damage potential, Reproducibility, Exploitability,
Affected users, Discoverability
DRM Data Reference Model
DRM Digital Rights Management DSS Data Security Standard
DTIC Defense Technical Information Center EAL Evaluation Assurance Levels
ENST Telecom ParisTech
EPRI Electric Power Research Institute
ERIM Erasmus Research Institute of Management ESOPE Evaluation de la Sécurité Operationnelle ESM Evaluator’s Scoring Metrics
EU European Union
FAQ Frequently Asked Questions FBI Federal Bureau of Investigation
FCD Final Committee Draft
FDCC Federal Desktop Common Configuration FEA Federal Enterprise Architecture
FIPS Federal Information Processing Standard FIRST Forum of Incident Response and Security Teams FISMA Federal Information Security Management Act FITSAF Federal Information Technology Security
Assessment Framework
FWP Sixth Framework Program
FWP7 Seventh Framework Program GIAP GIG IA Portfolio program GMU George Mason University
GNOSC Global Network Operation and Security Center GOTS Government Off the Shelf
GP Generic Practices
GPRA Government Performance Results Act GQIM Goal, Question, Indicator, Methodology
HIPAA Health Insurance Portability and Accountability Act I3P Institute for Information Infrastructure Protection
IA Information Assurance IA-CMM IA Capability Maturity Model
IAM INFOSEC Assessment Methodology
IASET Information Assurance Science and Engineering Tools IASM Information Assurance and Security Management IATAC Information Assurance Technical Analysis Center IATRP INFOSEC Assurance Training and Rating Program IAVA Information Assurance Vulnerability Alert
ICT Information and Community Technologies IDART Information Design Assurance Red Team IDS Intrusion Detection Systems
IEC International Electrotechnical Commission
IG Inspector General
INFOSEC Information Security
INFRES Institut TELECOM Computer Science and Networking Department
IORTA Information Operational Red Team Assessment IPS Intrusion Protection Systems
IRC Information Security Research Council ISA International Society of Automation
ISECOM Institute for Security and Open Methodologies ISMS Information Security Management Systems ISO International Organization for Standardization ISOT Information Security and Object Technology ISP Internet Service Provider
ISSA Information Systems Security Association
ISSEA International System Security Engineering Association ISSRR Information Security System Rating and Ranking
IT Information Technology
ITSEC Information Technology Security Evaluation Criteria ITUA Intrusion Tolerance by Unpredictable Adaption JCIAC Joint Council on Information Age Crime
JMRR Joint Monthly Readiness Reports JPL Jet Propulsion Laboratory
JQRR Joint Quarterly Readiness Reports
KGI Key Goal Indicators
KPI Key Performance Indicators
JTF-GNO Joint Task Force Global Network Operations
LC Loss Controls
LOE Level of Effort
McDiD Metrics and Controls for Defense-in-Depth MAIS Major Automated Information Systems MDAP Major Defense Acquisition Programs
METF Mean Effort to Security Failure MHS Military Health System
MOA Memorandum of Agreement
MSRAM Maritime Security Risk Analysis Model MTBF Mean Time-Between-Failure
MTTR Mean Time-to-Repair
NASA National Aeronautics and Space Administration NCSD National Cyber Security Division
NCSS National Computer Security Survey NDIA National Defense Industrial Association NII Network and Information Integration NIPP National Infrastructure Protection Program NIST National Institute of Standards and Technology NMCI Navy Marine Corps Internet
NRL Naval Research Laboratory NSA National Security Agency NSF National Science Foundation
NSTAC National Security Telecommunications Advisory Committee NVD National Vulnerabilities Database
OASD Office of the Assistant Secretary of Defense OECD Organization for Economic Cooperation and
Development in Europe OJP Office of Justice Programs
OMB Office of Management and Budget OpSec Operational Security
OSD Office of the Secretary of Defense
OSSTMM Open Source Security Testing Methodology Manual OVAL Open Vulnerability and Assessment Language
OVAL-ID Open Vulnerability and Assessment Language Identifier OWASP Open Web Application Security Project
PA Process Areas
PEPA Performance Evaluation Process Algebra PERFORM Performability Engineering Research Group PLA Protection Level Agreements
POA&M Plan of Action and Milestones
PP Protection Profile
PRM Performance Reference Model
PSM Practical Software and Systems Measurement Support Center
QoS Quality of Service
QUERIES Quantitative Evaluation of Risk for Investment Efficient Strategies
RAI Resiliency Assurance Index
RASQ Relative Attack Surface Quotient RAV Risk Assessment Value
R&D Research and Development
RDX R&D Exchange
ReSIST Resilience for Survivability in IST ROI Return on Investment
ROSI Return on Security Investment RTWF Red Team Work Factor
SAMATE Software Assurance Metrics and Tool Evaluation SANS SysAdmin, Audit, Network, Security
SCADA Supervisory Control and Data Acquisition SCAP Secure Content Automation Protocol SCARE Source Code Analysis Risk Evaluation SDLC Software Development Life Cycle
SEAS Structured Evidential Argumentation System SecLab Security Lab
SecMet Security Metrics Consortium
SEPG Software Engineering Process Group
SERENITY System Engineering for Security and Dependability
SG Security Group
SIG Special Interest Groups SLA Service Level Agreement SLE Single Loss Expectancy
SM Security Management
SOAR State of the Art Report
SOP Standard Operational Procedures
SP Special Publication
SPMO Security Project Management Officers SPP Security and Privacy Profile
SQUALE Security, Safety, and Quality Evaluation for Dependable Systems
SRD SAMATE Reference Dataset
SRM Service-Component Reference Model SSAA System Security Authorization Agreement
SSE CMM System Security Engineering Capability Maturity Model S&T Science and Technology
ST Security Target
ST&E Security Test and Evaluation
STEM Security Testing and Engineering Using Metrics S-Vector Scoring Vector
SwA Software Assurance (SwA)
TA Technical Alerts
TAF Trusted Agent FISMA
TCSEC Trusted Computer System Evaluation Criteria
TMA TRICARE Management Activity
T-MAP Threat Modeling framework based on Attack Path Analysis TOE Target of Evaluation
TRM Technology Reference Model TSABI Top Secret and Below Information
TSF Tolérance aux Fautes et Sûreté de Fonctionnement Informatique
TTOA Technical Target of Assessment
UK United Kingdom
UML Unified Modeling Language USAF United States Air Force
US-CERT United States Computer Emergency Response Team USMC/MCNOSC United States Marine Corps/Marine Corps
Network Operations and Security Command USSTRATCOM/
JTF-GNO United States Strategic Command/Joint Task Force Global Network Operations
VA/RM Vulnerability Assessment/Risk Management
VFT Value-Focused Thinking
VMS Vulnerability Management System VPN Virtual Private Network
VTT Valtion Teknillinen Tutkimuskeskus WISSSR Workshop on Information Security System
Scoring and Ranking
WG Working Group
XCCDF Extensible Configuration Checklist Description Format
YTD Year to Date