Also, Joaquin Crespo from the Spain contingent of OWASP contributed a full translation of the OpenSAMM 1.0 overview presentation. That’s also available on the downloads page.

To everyone involved in the translation work, I would like to personally extend my thanks and gratitude to each one of you for this valuable contribution to the project. If anyone reading this would like to lead a translation to your language of choice, just post a message to the SAMM mailing list and we’d be glad to help you get started.

You can read more about the file and download it on IPsec.pl at http://ipsec.pl/node/967

The main thing I want to share now is an activity-level mapping of the ~110 BSIMM2 activities to the corresponding 72 activities in SAMM. Obviously, this means that in some cases, more than one BSIMM activity may be mapped to a single SAMM activity. That being said, the overlap spots seem to make sense when we (the ~10 people that worked on it) looked at them in detail. Don’t take our word for it, though, please do review and send any feedback (mailing list or just comment below). And before you ask, yes, you probably will have to go read the respective BSIMM and SAMM activity descriptions in order to see the linkage for some of them (given the occasionally imprecise nature of written language, it’s not always obvious from the activity names alone).

It’s worth noting that we did leave two BSIMM activities unmapped. They are SM 3.2 “run external marketing program” and T 3.3 “host external software security events”. Based on the experience of the working group participants, these activities did not appear to directly improve an organization’s software assurance posture, rather, they appeared to be evidence that the organization was using its (presumably mature) software assurance posture to bolster its public perception or generate additional value in the business. Again, this is totally up for debate if anyone has an argument the other way, so please do share your thoughts.

Last, but certainly not least, I’d like to thank all the people at the Summit for the detailed and thoughtful conversations about using SAMM and about what we can do to make it even better.  Specifically, those that contributed and helped review this mapping (in no particular order):

• Colin Watson
• Seba Deleersnyder
• Steven van der Baan
• Bart De Win
• Justin Clarke
• Dan Cornell
• Sherif Koussa
• Brian Chess

The Problem

We have a pervasive problem in our field. We lump two disparate classes of security weakness together. Some articulate the difference as “business logic” vs. “technical” or “semantic” vs. “syntactic”. I’d like to build on a familiar term to developers: “domain”. Each kind of software weakness is domain-specific or domain-agnostic (or both). Making this distinction is critical. We currently try to fix both sets of weaknesses with the same sets of tools, processes, and expertise when in fact they need to be treated differently. Our current approaches to application security work well for domain-agnostic vulnerabilities, but we need domain experts to solve domain-specific problems.

Examples

Let’s suppose an energy wholesaler in a deregulated market builds a new web application. They hire security testers to perform a combined source code review and penetration test against their application prior to deployment. The testers find a series of well known web application vulnerabilities: cross-site scripting (XSS), insufficient password complexity requirements, user enumeration in forgotten password, direct object access, and the like. Now suppose this system had another vulnerability: by simply changing the date field on one of the pages to a day in the past, users have access to pricing history which may give energy market speculators an unfair advantage tantamount to insider trading. The odds of the testers finding this particular vulnerability rests squarely on how much they understand the domain of deregulated energy markets. XSS, direct object access, and similar vulnerabilities are domain-agnostic; they’re dangerous but they’re also well understood by the security community. The insider trading vulnerability could amount to millions or more in losses and is likely to go uncaught for a longer period of time.

Another example helps illustrate this dichotomy. Suppose a large manufacturer hires a seasoned penetration tester to assess the security of their web-based ERP application (think SAP, PeopleSoft, or Oracle Financials). The tester is likely to find a directory traversal vulnerability but much less likely to find segregation of duties violations. For example, suppose an end user could both authorize and receive a payment from the Accounts Payable module. If the tester understands basic accounting concepts, he may catch this. More complex accounting issues will likely remain undiscovered, however, unless the tester has deep domain knowledge of accounting.

Finally, consider a popular consumer electronics retailer who keeps stock-keeping unit (SKU) entries in its inventory database for all products that it carries. Typically, these entries might include product descriptions or model numbers for easy identification. Now, imagine that this retailer creates placeholder inventory entries in anticipation for a major product release from a giant technology company known for its secrecy around its future products. If these entries are overly descriptive and include legitimate details about the unreleased products, this information could be leaked through employees of the retailer who have access to the inventory database. Such a leak can potentially damage the business relationship that the retailer has with its partner.

Do You Speak Ubiquitous?

Eric Evans presents an in-depth treatise of the effects of the domain on software design in his book Domain-Driven Design (DDD). Evans presents a number of concepts related to DDD. One such concept, “Layered Architecture”, breaks software into four common layers: User interface/presentation, application, domain/model, and infrastructure. The book concentrates largely on the domain/model layer by presenting a series of modeling techniques and patterns. The “ubiquitous language” is a domain-specific language that evolves over time. In the energy wholesale example, developers may describe their model with terms like “market participant” and “congestion revenue rights” which are part of their ubiquitous language. Without understanding the concepts behind the ubiquitous language, security assessors run the risk of missing important vulnerabilities in the system. Developers, quality assurance testers, in-house security experts, and security experts with extensive expertise in a specific industry are often comfortable with the ubiquitous language. These domain experts are ideally suited to identify domain-specific threats during requirements and design and hunt for domain-specific vulnerabilities during development, testing, and deployment. XSS and other forms of domain-agnostic vulnerabilities should be relegated to domain-agnostic testing tools such as static analysis, automated security runtime testing, and integration of security testing into QA testing frameworks with human oversight. Third party penetration testers need not understand the domain very well to find these vulnerabilities. Parameter manipulation, cross-site request forgery (CSRF), fine-grained authorization bypass, manipulation of page-flow navigation, and other domain-specific vulnerabilities should be treated separately; they need people to think of them to find these vulnerabilities. If you elect to use the same people and tools to find both categories of vulnerabilities then you need to ensure the security experts take the time to understand their domain and speak the ubiquitous language. Having a deficiency of skill-set in either domain-agnostic vulnerabilities or domain-specific knowledge will almost certainly leave your system open to risk.

A Familiar Pattern

The security community explicitly understands the importance of domain-specific knowledge in technical domains. For example, many security experts have examined Secure Socket Layer (SSL), but presumably few take the time to understand the Online Certificate Status Protocol (OCSP). OCSP is a sophisticated replacement for Certificate Revocation Lists (CRLS); essentially a mechanism to determine if a particular certificate is revoked and no longer valid. In 2009, Moxie Marlinspike discovered that a man-in-the-middle could circumvent OCSP responders in certain cases simply be sending the number 3 [1]. The Request For Comments (RFC) for OCSP dates back to 1999 [2] – ten years before Marlinspike’s discovery. In 2008, Dan Kaminsky found a critical flaw in the Domain Name Service (DNS) which resulted in a massive multi-vendor patch [3]. In 2007, Brad Hill discovered that servers implementing the XML Digital Signatures protocol may be vulnerable to critical remote command execution vulnerabilities – particularly worrisome because XML Digital Signatures is part of the WS-Security specification [4]. In all of these cases, the security researcher took the time to understand the underlying protocol in some level of depth. Although they neither wrote the protocols nor created products that that adhered to the protocols, they understood the domain well enough to point out logic flaws.

Moving Forward

Identifying the distinction between domain-specifics and domain-agnostics is an important first step. The next step is to change the way we define application security techniques. We should approach both sets of problems with a commensurate set of techniques. We should divide Top N lists and lists of vulnerabilities into the two categories. We should correctly and clearly classify new vulnerabilities as being domain-specific, domain-agnostic, or both. Organizations should have separate plans to deal with both: for example, heavy reliance on automated testing and cross-industry security expertise for domain-agnostic vulnerabilities and more reliance on manual testing and domain understanding for domain-specific vulnerabilities. We should pool expertise in individual domains, perhaps by starting with particular industries, to share knowledge of domain-specific vulnerabilities and disseminating that knowledge as successfully as we currently disseminate domain-agnostic vulnerabilities. If cross-industry sharing is not viable, organizations should separate and maintain domain-specific security knowledge internally and share that information with developers, QA, and all stakeholders in the secure SDLC. Cross-industry security groups such as FS-ISAC for financial services do facilitate some security knowledge transfer, but they rarely share knowledge to the granularity of domain-specific threats against particular types of applications.

What Can You Do?

You can help make this happen. Start to explicitly talk about the difference between these two kinds of vulnerabilities in your daily conversations, and make sure that you adjust your techniques for detection, prevention, and remediation accordingly. Encourage your industry groups to start focusing on sharing domain-specific threat data. Are you aware of an attempted or actual incident on an application that involved manipulation of that application’s business logic? Find ways to share that data with your industry! Make the data anonymous if necessary, but find a way to centralize and share the information so that everyone benefits from preventing opportunities for fraud.

[1] http://www.blackhat.com/presentations/bh-usa-09/MARLINSPIKE/BHUSA09-Marlinspike-DefeatOCSP-PAPER2.pdf
[2] http://www.ietf.org/rfc/rfc2560.txt
[3] http://news.cnet.com/8301-1009_3-9998906-83.html
[4] https://www.isecpartners.com/files/XMLDSIG_Command_Injection.pdf

If anyone else has translated the presentation to other languages for local chapter presentations, please feel free to send them to me (or the mailing list) and we’ll get them posted for all to access.

I’ll post slides and presentation details after we’ve got them finished. But it should be a good morning, and for those in Australia wanting to know more, this is a great session for free to come along to.!

Yesterday Part 2 described how to create and validate the source roadmap data files. In this post, transforming the data files into the final SVG format we saw in Part 1 will be described.

The great thing about having data in XML is the ability to translate it easily into another format using Extensible Stylesheet Language Transformations (XSLT).

Transformation

Download the archive of files.

With Brenda’s assistance we now have an XSLT file which can be used to translate a valid and well-formed SAMM Roadmap source data file into a valid SVG file. You need the two XSL files included in the archive download linked above:

SAMM-1.0-roadmap-0.6-en_US.xsl
SAMM-1.0-utilities-0.1.xsl

You also need the base US English XML file (version 0.3) for SAMM v1.0 and your XML data files. The archive includes the two example XML files from Part 1 (again) and the US English SAMM base XML file. Remember the latter should normally be downloaded from the SAMM download page. Place all the files in a new directory.

Take a tool which can undertake XSLT conversions, and apply SAMM-1.0-roadmap-0.6-en_US.xsl to your XML file or one of the examples. An SVG file should be created.

Partial screen capture (as a JPEG) of a generated SVG SAMM Roadmap Chart

And that’s it. Alter your XML files to see the effect on the generated SVG charts.

If you don’t want to use XML and XSLT, you can of course just edit the SVG files directly using some sort of text editor.

Next

I hope to spend some time creating SVG charts for the SAMM scorecard charts sometime soon.

Part 1 demonstrated the final generated SCG charts. Here we will look at the starting point—the source roadmap data files.

The roadmap charts describe changing level data across a number of implementation states. The charts are not project planning tools and therefore the durations are neither defined nor indicated in the widths on the charts.

Maturity Level information can be 0, 1, 2 and 3 or where there is additional assurances in place beyond those indicated by the Level, the “+” symbol can be used so 0+, 1+ and 2+ are also allowed if required. For charting purposes these are treated as ½, 1½ and 2½. There is no assumption that Maturity Levels will increase through subsequent states; Maturity Levels can fall as well as rise, or remain static.

States

“Roadmaps … consist of [states] (the vertical bars) in which several Practices are each improved by one Level. Therefore, building a roadmap entails selection of which Practices to improve in each planned [state].” SAMM v1.0

Unlike in the previous spreadsheet version, the number of states (phases, steps or stages) which can be charted is flexible from 2 to 10. The greater the number of states there are, the wider the final generated chart will be. We will see below that your “states” can be called anything you want.

Format

Download the archive of files.

The archive contains two example SAMM Roadmap XML files. The primary structure of the XML files is:

<?xml version="1.0" encoding="iso-8859-1"?>
<maturity>
	<title>...</title>
	<states>
	...
	</states>
</maturity>

where the title (XML encoding) is used as a heading on the chart legend, and the Maturity Level data are included between the <states></states> markup tags.

The first markup within the section must be the Maturity Levels at the start i.e. state 0 (zero). State 0 has a title (“Start” in the example below) and description, but these are not used or displayed. All security Practices that are to appear on the roadmap must be defined within the <levels> markup.

<state number="0">
	<title>Start</title>
	<description></description>
	<levels>
		<level security-practice="SM" value="1" />
		<level security-practice="PC" value="0" />
		<level security-practice="EG" value="0" />
		<level security-practice="TA" value="0" />
		<level security-practice="SR" value="0" />
		<level security-practice="SA" value="0" />
		<level security-practice="DR" value="0" />
		<level security-practice="CR" value="1" />
 		<level security-practice="ST" value="0" />
		<level security-practice="VM" value="0" />
		<level security-practice="EH" value="0" />
		<level security-practice="OE" value="0" />
	</levels>
</state>

The values for the attribute “security-practice” must match the security Practice attribute “id” defined in the <security-practice> markup tag within the base SAMM XML file (e.g. SAMM-1.0-XML-0.3-en_US.xml mentioned in Part 1). “SM” is “Strategy & Metrics”, “PC” is “Policy & Compliance”, etc.

Subsequent state numbers (1, 2, 3, etc) must include values for the title, description, and as mentioned in Part 1, only data for Practices where the Maturity Level changes should be included:

<state number="1">
	<title>Phase 1</title>
	<description>2010/11 Michaelmas Term</description>
	<levels>
		<level security-practice="EG" value="1" />
		<level security-practice="SR" value="0+" />
 		<level security-practice="ST" value="1" />
		<level security-practice="VM" value="0+" />
	</levels>
</state>
<state number="2">
	<title>Phase 2</title>
	<description>2010/11 Hilary Term</description>
	<levels>
		<level security-practice="SM" value="2" />
		<level security-practice="EG" value="2" />
		<level security-practice="TA" value="1" />
		<level security-practice="DR" value="1" />
		<level security-practice="CR" value="2" />
 		<level security-practice="ST" value="2" />
		<level security-practice="OE" value="1" />
	</levels>
</state>

Subsequent stages are defined in the same manner. The file is saved with an XML extension.

For the eagle-eyed amongst you, you might have noticed a vertical dashed line in the SVG example shown in Part 1, which doesn’t appear in any of the roadmaps in the SAMM document. This is a new optional attribute which can be added to one of the stages. Just add the attribute “marker” with value “true” in one of the <state> tags and the line will be drawn. This might mean "status now" or an important event on the timeline, but that can be described in your accompanying text or presentation.

Validation

We tried the make the source data files as human-readable as possible, but thought we also needed to provide a way to validate the format.

Firstly check the file is well-formed. The simplest method is to use the W3C Markup Validation Service to upload the file or directly input the file content. If your data contains confidential information, you may want to undertake this check locally instead.

Then once the file has passed the previous check, test the XML structure against the XML Schema Definition (XSD) provided in the download link above:

SAMM-1.0-roadmap-0.3.xsd

You will need some sort of XML tool for this. The XSD includes some assertion statements, and therefore needs XSD v1.1 enabled in a Saxon parser. It also has some Schematron statements which should be validated as well.

Now you should have validated XML files.

Continues…

In Part 3 tomorrow, the automated process for transforming the source data into the final SVG files will be described.

OpenSAMM documentation has a big design wow factor and SAMM users, such as development companies, auditors and security consultants, are often keen to mimic the style, especially the graphs. So, last year I developed a Microsoft Excel spreadsheet to help in the creation of roadmap charts.

However it was a bit of a fudge to make the output look similar to the Adobe Illustrator versions in SAMM. The spreadsheet was difficult to alter if you had more than four stages in the roadmap, it was restricted to whole level values (not 0+, 1+ or 2+), and it included background images making it fiddly to edit.

A better way?

Open Office Calc didn’t seem to provide enough functionality to generate the charts, and Microsoft Visio is quite expensive, and even then I couldn’t make it format the data as required. Last year I was invited by David Rook to present SAMM at an OWASP Ireland chapter meeting in Dublin. After the presentation, I spoke with Eoin Keary (OWASP Board member) and we discussed whether a server-side option could be built which would be more flexible. But I had begun using Scalable Vector Graphics (SVG) for another project and realised these would be a better solution for creating the charts, since modern web browsers support SVG markup directly and it could all be created locally rather than using a remote service.

SVG charts

I began working on some mockups, and in March this year posted them to the SAMM mailing list, asking if anyone else had any XML Path Language (XPath) or Extensible Stylesheet Language Transformations (XSLT) experience with which to help improve my initial efforts. Brenda Larcom kindly replied. What a difference she made. Brenda’s initial ideas included:

• have a more explicit/human-readable connection between the state (phase, stage or step) and the target maturity levels for that state;
• use sparse data, i.e. if a level doesn’t change in a state, the chart uses the data from the previous state; and
• segregate data (put the data in elements) from display options (put options in attributes).

We spent the next couple of months bouncing ideas and files back-and-forth, and Brenda’s expertise meant we ended up with a much more compact file structure. SAMM project leader Pravir Chandra also helped by providing all the textual information from SAMM in an XML file. The base US English XML file (version 0.3) for SAMM v1.0 is currently available (SAMM-1.0-XML-0.3-en_US.xml in a Zip archive on the SAMM download page, adjacent to the PDF version). Save that—you’ll need it in Part 3.

With one thing and another, it’s been a little while since then, but Pravir asked if we could share our efforts via this blog rather than using the mailing list alone.

I’d like to begin at the end, and show you examples of the generated SVG files.

Examples

Download the archive of files.

We need an external style sheet (included in the download above) to view the generated SVG files. Extract SAMM-1.0-roadmap-0.4.css into a directory, for example:

e:\path\samm\roadmap-part-1

Then extract the two example SVG files into the same directory. If you view the source of the SVG files, you will just see the XML content like this partial view below:

Example SAMM Roadmap Chart Source Code

You can use your web browser to render the chart visually. Open each SVG file in turn from the directory, for example:

file:///e:/path/samm/roadmap-part-1/roadmap-ex1.svg

You should see something like the JPEG screen capture below:

Example Rendered SAMM Roadmap Chart

The charts are of a fixed size, but you can alter your browser zoom level. If you want a static file, take a screen capture the image and paste it into your report. The external style sheet file is referenced from within the SVG file like this:

<?xml-stylesheet type="text/css" href="SAMM-1.0-roadmap-0.4.css"?>

So if you change the CSS filename or location, you must edit the SVG file (perhaps just using a text editor). You can of course customise the CSS file to alter the chart appearance.

Continues…

In Part 2 on Monday, the XML format for source roadmap data will be discussed, and in Part 3 an automated process for transforming that data into the final SVG files, like the one above, will be described.

 I have conducted, sold and project managed SAMM engagements to financial service (FS) organisations throughout Europe over the past years. It is obvious the demand is growing for such services. The rise in demand of  “security@source” be it via code review, secure development or grey box penetration testing and a supporting framework to tie it all together and understandably so as SAMM is one of the first pragmatic benchmarking and assessment frameworks for the somewhat ancient “Security in the SDLC” challenge.  

The financial services industry is the perfect work stream for frameworks like SAMM. Financial services are widely known as the an area which invests heavily in areas such as information security, it’s heavily regulated (some say not heavily enough) and  a daily challenge to FS is to maintain leading edge security but manage costs and usability whilst also being compliant with industry regulations,  corporate governance and local/regional/global  legislation.

SAMM covers four domains which in turn have sub domains. These four “pillars” attempt to examine all aspects of software development, all external catalysts which may result in either making security more robust or result in weakness.

The beauty of SAMM is its simplicity:

It would be naive to assume SAMM is a “silver bullet” in terms of SDLC assessment but it is a very pragmatic solution to a rather complex ecosystem.

The questionnaire is simple and effective assuming knowledge of secure application development as it can sometimes be open to misinterpretation.  One of the key challenges in developing SAMM was delivery/authoring of the questionnaire. The risk being that individuals being questioned may misunderstand the questions.

The key to an accurate SAMM effort is [in audit speak] to procure a decent “sample space”.  Sample space is a function of the amount and the diversity [roles within the SDLC] of the individuals interviewed by the SAMM reviewer. Accuracy of the answers given is also important as you shall get divergence based on role within the organisation.

Road map definition is also a challenge and knowledge of what are the  focus points for the organisation being assessed are very important to develop the roadmap. [E.g. A financial services organisation may focus on regulatory and compliance issues but a software development house may not so much.] A principal benefit of SAMM is the ability to define a high level roadmap and drilling down into each activity in order to define what is required to reach the required SAMM level for a given domain.