Cyverity

Get Started

Cybersecurity Standards Scorecard: 2021 SANS Edition

Bearded Man Holding Laptop in Cybersecurity Office

Speaker: James Tarala
Event: SANS Webcast
Date: November 10, 2021
Watch on YouTube: https://www.youtube.com/watch?v=53WNguylLRo 

Introduction 

The Cybersecurity Standards Scorecard provides an annual review of leading security frameworks, evaluating their effectiveness, scope, and applicability. James Tarala, a senior instructor at SANS, presents the 2021 edition of the Scorecard, highlighting key cybersecurity standards and their impact on security programs. 

This webcast challenges the common belief that all cybersecurity standards are essentially the same. Instead, James Tarala demonstrates how each standard has unique strengths, weaknesses, and applicability based on an organization’s security needs. 

 

Key Takeaways 

  • Cybersecurity standards are not interchangeable. Each has a unique focus and should be selected based on organizational needs. 
  • Prioritization is key. Not all controls in a framework are equally important—organizations should focus on the most effective measures first. 
  • Mapping threats to controls is rare. Few standards explicitly link security controls to specific cyber threats. 
  • A governance-driven approach enhances security programs. Successful security programs integrate governance, operations, and technical controls. 
  • The cybersecurity landscape evolves. Keeping standards up to date is essential for relevance in modern threat environments. 

 

Summary of the Discussion 

Why Compare Cybersecurity Standards? 

With over 50 cybersecurity frameworks in existence, choosing the right one can be overwhelming. Tarala emphasizes that not all frameworks are created equal, and some are better suited for different use cases. 

While many organizations default to compliance-based frameworks, Tarala argues that effectiveness should be the primary criterion when selecting a security standard. The Cybersecurity Standards Scorecard provides an objective comparison to help organizations make informed decisions. 

 

The Evaluation Criteria 

James Tarala’s methodology for evaluating standards includes: 

  • Governance, Operational, and Technical Controls – Does the framework provide a balanced approach? 
  • Recent Updates – Is the standard actively maintained and reflective of current threats? 
  • Community-Driven Development – Does the standard allow open contributions and feedback? 
  • Threat Mapping – Does it explicitly connect security controls to known cyber threats? 
  • Applicability to Modern IT Environments – Can the framework be used in cloud, SaaS, and industrial control system environments? 
  • Prioritization of Controls – Does it guide organizations on what to implement first for maximum impact? 
  • Metrics and Measurement Guides – Does it provide tools for evaluating implementation effectiveness? 

Each standard is scored using a five-point grading system, with final letter grades assigned based on performance across all categories. 

 

Notable Framework Comparisons 

CIS Controls (Version 7.1 & 8) 

  • Strengths: Strong focus on technical controls and actionable security measures. 
  • Weaknesses: Limited privacy and governance guidance. 
  • Changes in Version 8: Increased focus on policy-driven controls and governance. 
  • Final Score: B+ (Version 7.1), B (Version 8 due to missing supplementary guidance at the time of evaluation).

 

NIST Cybersecurity Framework (CSF) 

  • Strengths: Well-regarded for governance and risk management. 
  • Weaknesses: Lacks clear technical security measures. 
  • Relevance: Declining industry adoption and lack of recent updates. 
  • Final Score: C+

 

Cybersecurity Maturity Model Certification (CMMC) 

  • Strengths: Designed for government contractors, integrates NIST 800-171. 
  • Weaknesses: Complex certification process and evolving requirements. 
  • Industry Interest: Rapid adoption, but recent Pentagon changes may impact implementation. 
  • Final Score: B-

 

ISO 27002 

  • Strengths: Strong governance and compliance focus, best for regulatory alignment. 
  • Weaknesses: Minimal technical security guidance, outdated approach. 
  • Software Security: Surprisingly strong secure software development coverage. 
  • Final Score: B-

 

PCI DSS 

  • Strengths: Clear compliance requirements for financial data security. 
  • Weaknesses: Minimal security innovation, compliance-driven rather than risk-based. 
  • Final Score: C+

 

HIPAA (U.S. Healthcare Security Rule) 

  • Strengths: Long-established regulation, foundational security principles. 
  • Weaknesses: Extremely outdated and offers little beyond basic security hygiene. 
  • Final Score: D+

 

COBIT (ISACA) 

  • Strengths: Highly focused on governance and IT risk management. 
  • Weaknesses: Lacks technical depth, limited real-world implementation. 
  • Final Score: C

 

MITRE ATT&CK & Enterprise Mitigations 

  • Strengths: Explicitly maps controls to real-world threats. 
  • Weaknesses: Not a traditional security framework—designed as a reference model. 
  • Final Score: B

 

Collective Controls Catalog (Research Initiative) 

  • Strengths: Aggregates over 40 security frameworks into a comprehensive control baseline. 
  • Weaknesses: New initiative, lacks widespread adoption. 
  • Final Score: A- (Due to being a research-driven initiative rather than a regulatory framework.)

 

Actionable Insights 

  • Choose the right standard for your needs. Avoid assuming all frameworks are equally effective. 
  • Balance governance, operations, and technical security. A well-rounded security program integrates all three. 
  • Use prioritization to drive security investments. Implement high-impact controls first. 
  • Demand better threat mapping in security frameworks. Organizations should advocate for explicit control-to-threat alignment. 
  • Continuously evaluate and adjust security strategies. Cyber threats evolve—so should your security approach. 

 

Conclusion 

The Cybersecurity Standards Scorecard highlights the diversity in security frameworks and the importance of selecting the right one for your organization. Rather than blindly following compliance mandates, organizations should prioritize effectiveness, governance, and technical security measures. 

By understanding the strengths and weaknesses of cybersecurity standards, security leaders can make informed decisions, strengthen their defenses, and align their security programs with business objectives. 

 

For more insights on this topic, watch the full webcast here.