I am getting really tired of listening to whining without posited solutions. Not only has the security subset of the blogosphere dried up over the last few months, but the whining seems to be increasing. Compliance has been the whipping boy du jour for most of the year, but risk assessment also appears to be back up for a beating this month. I think the worst part of it all is that the criticisms I've read typically lack the proper background research, or they end up being about other issues rather than being an attack on risk assessment itself.
There are several points that I want to discuss around these topics. First, from a regulatory perspective, we're still closer to living in the land of common law than we are to modern governed society. There are limits to how effective that can be. Second, we need to make sure that we focus our energies on valuating the right things. There's a lot of churn about how certain words or concepts aren't estimable or have no intrinsic value, but it's a red herring argument. Lastly, and perhaps more importantly, we need to realize that the reason we are where we are today in infosec is because of a disconnect because actions and consequences. We now know that this must change.
Limits and Strengths of Common Law
Common law in this context is talking about the evolved case law and social rules around which our society is governed. Much law today still relies on court decisions regarding fairness and prudence rather than a statutory or regulatory rule that must be enforced.
The problem that we've encountered with common law is that it has been slow to evolve to meet the needs of the information-centric society. Regulators and legislators have tried to start filling the gaps, but their processes move slowly. In essence, we've encountered a scalability issue with common law in that it simply can't keep pace with the technology evolution these past 25 years (or more).
In lieu of evolved common law, it is then incumbent upon key actors to establish regulations and drive compliance initiatives, either within industries or executive branch agencies. This approach, however, also has limits. For instance, an industry is likely to limit self-regulation to the minimum required to stave off statutory or regulatory laws. As such, industries are conflicted between doing what is necessary to adequately protect itself, versus doing the bare minimum to reduce outside scrutiny.
At the same time, the future of security will increasingly hinge on common law. Legal defensibility doctrine (also so here) informs us that organizations need to take proactive measures that build a defensible legal argument demonstrating (with evidence) that they have done what is necessary and reasonable to protect themselves and their assets. The legal system has thus far been inadequate to the task of enabling civil suits to pursue remedy against organizations that have been breached because of a failure to do what is reasonable, but that status is unsustainable. Organizations need to start preparing now for a future in which they will be held accountable by all parties with a vested interested (including shareholders, investors, employees, customers, and government regulators).
Risk and Intrinsic Value
The last couple weeks have seen a new onslaught of attacks on risk, risk assessment, and risk management. However, while these attacks have used these terms, their quips are really with other issues. Consider Anton Chuvakin's "mega-epiphany" that "compliance" has no intrinsic value. In reading through his post, it turns out that his realizations have nothing to do with compliance itself (in fact, he acknowledges the value of compliance). Instead, his "mega-epiphany" ends up attacking lousy risk assessment practices (ALE in particular). But, the problem isn't with risk assessment, per say, but rather with how we estimate values - particularly the value of information.
You'll note that Anton's post derives from Rich Mogull's Secure360 talk, which itself seems to correlate to his recent "FireStarter" thread "The Only Value/Loss Metric That Matters" (which also claims to attack ALE and risk assessment, but really ends up back on this question of the intrinsic value of information). This theme is echoed even louder in the (now tired) arguments of Donn Parker, as seen recently in the LinkedIn ISSA Group Discussion "An Introduction to Factor Analysis of Information Risk (FAIR)".
Before tackling the "value of information" question, let's first put to rest the value of compliance. Chris has a great post up entitled "On Greed and Complianciness" that tackles the need for regulations and compliance head-on. It's something that Rich Mogull has also said before. Simply put: if you don't require organizations to make changes, then they're going to take the least expensive route and not implement changes. From my perspective, this can be addressed in part through specific technical regulations, in part through laws requiring a legally defensible position, and in part through setting steep consequences that change the costs associated with these bad behaviors (see below).
Back to risk and the intrinsic value of information, we then have an interesting dilemma. Do words or data have any particular value on their own, completely out of context? Absolutely not. Context is everything. How, then, do we estimate the value of information? The answer is partly that it must be evaluated in context. Now, this is where things go wonky. Anton and Rich and Donn all seem to go down this "unknown unknowns" road where, because we don't have perfect data, we thus cannot make any decisions or estimate value at all. Nothing, of course, could be further from the truth.
As an example, take the Factor Analysis of Information Risk (FAIR) risk assessment methodology. FAIR (or FAIR originator, Jack Jones) acknowledges these challenges and then tackles them head-on through use of a taxonomy and statistical skills that help increase confidence in estimates and reduce the impact of uncertainty (in fact, Gunnar Peterson has a great blog post up, Playing Defense", that talks about using margins to address these concerns - which is comparable to the notion of using ranges and confidence, a la Hubbard's How to Measure Anything).
The key take-away here is that there is no such thing as "perfect" information, and thus we have to work with what we have. While there will always be uncertainty (those niggling "unknown unknowns" chief among them), there are ways that we can build-in margins to account for them, and there are ways to boost are confidence in and accuracy of those margins to better improve our estimates. These methodologies are in practice, have been demonstrated to work, and probably blow the average person's mind (Bayesian statistics certainly blows my mind).
In terms of information valuation, this is fundamentally a surmountable business issue. It hinges on an organization being able to know itself and its business well enough to create the necessary context for assessing the value of a key asset. Coca Cola clearly knows the value of it's secret Coke recipe. Should this not be universally true; that all businesses should know the inherent, intrinsic value of its own information?
Consequences Breed Reform
The last point I want to make here is that compliance and risk management is pointless unless there are real consequences for bad decisions or behavior. If organizations refuse to take a legally defensible position, then there must be a cost. That cost could be fines, law suits, loss of a business license, or any other of a number of ideas. However, the key point here is that consequences are vitally important to the effectiveness of spurring change.
There have been two articles/papers released this year already on the topic, and both have reached the same conclusion. Specifically:
* "Compliance with Information Security Policies: An Empirical Investigation" (IEEE Computer Magazine, February 2010 - vol. 43 no. 2, gated/for-pay)
* "Ticket or Click-It by Lance Spitzner of HoneyTech (and, of course honeypot/honeynet fame).
In the latter case, Spitzner reviewed a study that found that awareness, consequences, and enforcement were all interrelated and reinforcing. Increased awareness efforts led to stepped-up enforcement led to increased realization of consequences led to increased awareness led to decreased violations (and, subsequently, risk).
The point here is this: as much as we like to think that positive incentives (rewards) are preferable and useful for changing human behavior, the simple truth is that humans respond far better to consequences. It seems to derive from the tenet that change occurs for one of two reasons: because we really want to change, or because we're forced to change (such as through a traumatic experience). The bailout of Wall Street notwithstanding, it is imperative that bad decisions and bad behaviors result in negative consequences that cause organizations to learn and evolve. As with Darwin's theory of evolution, those that don't evolve should perish, plain and simple.
Final Thoughts
As per usual, there are no silver bullets. If there wasn't a challenge, then I don't think most of us would be in this field. Compliance and risk management are tools in our belts to help protect users, data, and organizations. Some of these tools have begun to mature quite nicely (e.g. FAIR), while other areas are still lagging behind (laws inevitably lag behind due to the amount of heavy-lifting required). Toward that end, we need to make consequences more real as we prepare for a new age where legal defensibility will become the new norm, and where the legal system will evolve to support this norm through common law, statutory law, and regulatory law. Until then, we need to get the message out that changes are coming and that consequences will be real and potentially severe. The free pass times are over and the storm clouds on the horizon look frothy with lightning. Let's hope that the first strike is realization and not obliteration.
Nice post. Why all the hullabaloo for such aging topics? Is it just time to discuss them again or am I missing something?
In my experience, dealing with intrinsic value is simple: engage the business owners. Security is good at understanding threats and conducting assessments to estimate likelihood. Take this information and collaboratively work with the CIO, head of Marketing, Legal, Sales, Product Development, CEO (depending on size) once or twice a year. Include evidence from your assessments, incidents, and peers. They'll tell you what's important.
The executive team owns risk tolerance decisions, security simply provides data and drives the process. No fancy formulas or technology required. It's only hard if you go it alone.
@Jared -
Thanks. I think that there's a lot of backlash against risk management these days because:
a) Wall Street had some bad failures,
b) people don't generally understand it and are increasingly inclined to fear+hate what they don't understand,
c) accepting risk management practices means having accountability, which also scares people.
These conversations will persist as long as we have vocal cynics spouting ignorance and causing confusion. Their ominous meddling belies their motives.
-ben