Cyverity

Get Started

Cybersecurity Standards Scorecard: 2024 RSA Edition

Cyber security team working in a Cyber Security Operations Center SOC to protect systems and

Speaker: James Tarala
Event: RSA Conference
Date: June 10, 2024
Watch on YouTube: https://www.youtube.com/watch?v=k3Lqbcf2Jpo 

 

Introduction 

The Cybersecurity Standards Scorecard is an annual evaluation of cybersecurity frameworks, assessing their relevance, effectiveness, and applicability. James Tarala, a senior faculty member at SANS, presents the 2024 RSA Edition, offering an in-depth comparative analysis of major cybersecurity standards and how organizations can use them to build more resilient security programs. 

Building on prior editions, this year’s Scorecard includes 94 security frameworks with an expanded focus on regulatory trends, governance models, and technical security control effectiveness. The goal of this study is to help security teams understand which frameworks best align with their organizational needs and to evaluate standards beyond mere compliance requirements. 

 

Key Takeaways 

  • Cybersecurity standards are not interchangeable. Each has unique strengths, weaknesses, and intended use cases. 
  • Prioritization of security controls is disappearing. Many frameworks list controls without providing clear guidance on implementation sequencing. 
  • Regulatory frameworks like NYDFS and SEC cybersecurity rules are reshaping compliance expectations. Organizations must adapt their governance and risk management strategies accordingly. 
  • DevSecOps, SaaS, and serverless architectures are underrepresented in most standards. Many frameworks still focus on traditional infrastructure, despite industry shifts. 
  • ISO 27002 and NIST 800-171 are gaining popularity as organizations seek structured, third-party risk management solutions. 

 

Summary of the Discussion 

The Complexity of Cybersecurity Standards 

Organizations today face a growing number of cybersecurity frameworks, making it difficult to select which standards to follow. Tarala challenges the idea that all cybersecurity frameworks are the same, emphasizing that each serves a distinct purpose. 

To improve clarity, the 2024 Scorecard introduces a new classification system, grouping frameworks into four categories: 

  • Comprehensive Cybersecurity Frameworks – Broad, all-encompassing security models (e.g., NIST CSF, ISO 27002, CIS Controls). 
  • Regulatory & Compliance Frameworks – Standards tied to legal or industry-specific compliance (e.g., PCI DSS, HIPAA, CMMC 2.0). 
  • Cyber Hygiene Standards – Government-driven best practices that outline basic security principles (e.g., ACSC Essential Eight, NSA Top 10). 
  • Governance & Risk Management Standards – Focused on security program management and oversight (e.g., COBIT, AICPA TSC, ISO 27001). 

Evaluation Criteria 

Each framework is assessed using 12 key criteria, including: 

  • Governance, Operational, and Technical Controls – Does the framework provide a balance of policy, implementation, and technical guidance? 
  • Recent Updates – Is the framework actively maintained to reflect modern threats? 
  • Community-Driven Development – Can organizations provide feedback and contribute to updates? 
  • Threat Mapping – Does the standard explicitly link controls to known cyber threats? 
  • Applicability to Cloud and DevSecOps Environments – Does it support modern architectures and emerging security challenges? 
  • Prioritization of Controls – Does it provide guidance on which controls should be implemented first? 
  • Metrics and Measurement Guides – Does it include tools to assess implementation effectiveness? 

Each framework is graded on a five-point scale, with letter grades assigned based on overall performance. 

 

Notable Framework Comparisons 

NIST Cybersecurity Framework (CSF) – Version 2.0 

  • Strengths: Improved governance structure, widely recognized. 
  • Weaknesses: Lacks detailed technical controls, no explicit threat mapping. 
  • Industry Adoption: Increasing use in private-sector organizations. 
  • Final Score: B

 

ISO 27002:2022 

  • Strengths: Strong governance and compliance focus, widely used for third-party risk management. 
  • Weaknesses: Limited technical security controls, minimal focus on modern threats. 
  • Final Score: B- 

 

CMMC 2.0 (NIST 800-171 & 800-172) 

  • Strengths: Clear compliance structure, strong technical control coverage. 
  • Weaknesses: Ongoing regulatory uncertainty, frequent certification changes. 
  • Industry Adoption: Expanding beyond Department of Defense contractors. 
  • Final Score: B+ 

 

CIS Controls (Version 8) 

  • Strengths: Actionable security measures, practical technical guidance. 
  • Weaknesses: Limited governance and privacy considerations. 
  • Final Score: B 

 

PCI DSS 4.0 

  • Strengths: Well-defined compliance structure, effective for financial data protection. 
  • Weaknesses: Primarily compliance-driven, lacks modern threat adaptation. 
  • Final Score: C+ 

 

HIPAA (U.S. Healthcare Security Rule) 

  • Strengths: Foundational for healthcare security. 
  • Weaknesses: Outdated, lacks specific technical guidance. 
  • Final Score: D+ 

 

COBIT (ISACA) 

  • Strengths: Strong IT governance model. 
  • Weaknesses: Lacks prescriptive security controls, difficult to implement without supplementary frameworks. 
  • Final Score: C 

 

MITRE ATT&CK & Enterprise Mitigations 

  • Strengths: Explicitly maps controls to real-world cyber threats. 
  • Weaknesses: Not a comprehensive security framework—designed for reference rather than implementation. 
  • Final Score: B 

 

Collective Controls Catalog (Research Initiative) 

  • Strengths: Aggregates over 90 security frameworks into a comprehensive baseline. 
  • Weaknesses: Still new, lacks broad industry adoption. 
  • Final Score: A- 

 

Actionable Insights 

  • Align security frameworks with business needs, not just compliance mandates. Compliance does not always equal effective security. 
  • Balance governance, operations, and technical security controls. A strong security program integrates all three. 
  • Push for better threat mapping in security frameworks. Organizations should advocate for explicit control-to-threat alignment. 
  • Prioritize security investments. Implement high-impact controls first to maximize risk reduction. 
  • Adapt security strategies to evolving technology environments. DevSecOps, serverless computing, and SaaS security require dedicated security frameworks. 

 

Conclusion 

The Cybersecurity Standards Scorecard provides a data-driven analysis to help security leaders choose the right security frameworks based on effectiveness, governance capabilities, and technical coverage. 

By understanding the strengths and weaknesses of cybersecurity standards, organizations can align their security programs with business objectives and real-world threats—ensuring proactive risk management rather than reactive compliance. 

 

For more insights on this topic, watch the full webcast here.