Let's create guidance for management and auditors concerned about DevOps.
See all
Members (630)
Gene Kim's profile photo
Antony Ryan's profile photo
Aaron Weaver's profile photo
Abe Hassan's profile photo
Adam Barrett (Utahcon)'s profile photo
Adam Behn's profile photo
Angela Plater's profile photo
Adam Malone's profile photo
Adele Green's profile photo
Andy Brown's profile photo
Aleksandar Ristic's profile photo
Balaji Vajjala's profile photo
Alex Knorr's profile photo
Ariel Zavala's profile photo
Auction markete's profile photo
Andy Cheng's profile photo
Anna Kennedy's profile photo
Amo Chumber's profile photo
Artur Martins's profile photo
Andrew Waterman's profile photo
Andy Lole's profile photo
Andrew McKenzie's profile photo
Andrew Stangl's profile photo
Anh Doan's profile photo

Stream

Join this community to post or comment

Logentries

Discussion  - 
 
DevOps pros -- we're trying to find out more about approaches to debugging. Mind taking this short survey and letting us know your thoughts? We'll share the results across social media. Thanks!
 
trevp.me/devops-survey
1
Add a comment...

Gene Kim
owner

Discussion  - 
 
+James D and I presented at #Agile2014  this week on the DevOps Audit Defense Toolkit, in a presentation called "Keeping The Auditor Away: DevOps Audit Compliance Case Studies."

http://www.slideshare.net/realgenekim/keeping-the-auditor-away

PS:  Please get your comments to the Toolkit in by Fri Aug 8.  We'd like to close out the review process by then.  Thank you!
GenOrganizations and development teams are moving beyond waterfall models to those embracing a continuous delivery/DevOps-style set of processes. The deployment of doing tens, hundreds, or even thousands of deploys per day as 'normal' does not align to the SDLC, separation of duties, and common controls expected by auditors. In this presentation, we will describe what auditors look for in a compliance audit, how to develop alternate control proce...
7
1
Elizabeth Lawler's profile photo
Add a comment...

Gene Kim
owner

Discussion  - 
 
Here's a blog post I wrote about scoping the IT portions of audits correctly.  It explains most of John's miraculous transformation in "The Phoenix Project", where he goes from a bottom-up control philosophy, to a top-down risk based philosophy.

One of the underpinning tools for this is GAIT and GAIT-R, which we developed as part of a task team at the Institute of Internal Auditors in response to SOX in 2005. I had the privilege of working with +Norman Marks and Ed Hill on codifying how to correctly scope the IT portions of SOX audits.

The DevOps Audit Defense Toolkit deals not so much with scoping, but with designing and demonstrating the effectiveness of controls.  But between "scoping" and "substantiation", we've covered most of the skills required to plan and execute effective audits

http://itrevolution.com/audit-101-for-devops-resource-guide-for-the-phoenix-project-part-3-correctly-scoping-it-using-gait-and-gait-r/
Bad things happen to organizations when we scope incorrectly the IT portions of an audit, especially for DevOps, as illustrated in "The Phoenix Project."
6
Add a comment...
 
More impressive work from +James Wickett on integrating infosec into continuous delivery and #devops  work patterns.  It talks about the mismatch of DevOps and infosec, and how to bridge them with engineering practices and technologies.

 http://www.slideshare.net/wickett/attacking-pipelinessecurity-meets-continuous-delivery
Talk given at ISC2 Secure SDLC event in Austin, TX The release velocity for our applications is increasing, often leaving security testing behind. In some cases, the security team ends up being the bottleneck. That's bad. In an idyllic world, security testing would happen earlier in the development lifecycle, but lets do one better. Lets do security testing on every code change. Using automation tooling and DevOps practices, this talk will help y...
3
1
James Kwong's profile photo
Add a comment...

Gene Kim
owner

Discussion  - 
 
All, ,thank you for all the comments on the early draft!  As you might have seen, we've been responding to the comments, integrating new text into the draft.

Due to various other commitments, we've taken a 2 week hiatus.  We'll resume work on it next week.

PS: George Hulme wrote a great article on the toolkit here: http://devops.com/features/devops-getting-past-audit/
When subject to an audit DevOps process can be challenging for auditors to understand, it is important that process that track everything are used.
2
Add a comment...
 
Some good thoughts in here about compensating controls in a DevOps environment. Other ideas not addressed in the article would be code reviews and automated testing (of all forms).
2
Add a comment...

Gene Kim
owner

Discussion  - 
 
All -- +Jeff Gallimore +Byron Miller +James D and I have all been working on the document. Expect reviewable document on May 14th. I'm so excited about the work we've done.  The team is working, and the work is good.  :)

Stay tuned.
6
Gene Kim's profile photo
3 comments
 
+Chris Price Yes, please feel free to share with your auditors -- we'd love to hear what they think, and if they have any concerns or objections that we should have put into the doc.  Thanks!  --Gene
Add a comment...

James D

Discussion  - 
 
This article highlights that in continuous develop/deploy environments there are systems, such as Jenkins, to integrate with other systems (hosted and local).

Challenge: To provide some level of assurance, an auditor would need to define these connections and specify a level of trust.

Request: Are teams considering sandboxing their DevOp activities to linkable (and able to document/whiteboard ondemand) systems, or are teams simply not leveraging such abstracted systems?

Curious on ideas on this premise, the reality, and any activities.
3
Kevin Gilpin's profile photo
 
Here's another post (mine) about using Jenkins in a critical develop/deploy environment. We apply comprehensive access control, and capture a detailed audit of Jenkins interactions with end-users and with system credentials.

http://www.conjur.net/blog/2014/03/06/conjur-jenkins.html
Add a comment...

Mike Kavis

Discussion  - 
 
Been through 2 SOC2 and 1 HIPAA audit on 100% AWS implementations (2 different companies). Interested to particpate in discussions here.
4
Chris Hoey's profile photoMike Kavis's profile photo
4 comments
 
I have not used it yet
Add a comment...

John Cooper

Discussion  - 
 
A fictional story about how one company transforms it's it operations using modern agile processes. Taking on audits and failures in the mass of business changes. It's not the best story you will read this year but it does describe some interesting approaches and changes. Worth a read just to get you thinking about the concepts without being another dry tech process book.
I found it worth the read. Maybe a little annoying in how simple it made things sound but it's an easier read than an ITIL manual!
3
Jeff Gallimore's profile photoGene Kim's profile photo
3 comments
 
Haha.  Thanks so much for the kind words!  :)  Now we've got to capture/codify what is required to make the auditors not freak out when they see DevOps-style control environments.  :)
Add a comment...
 
More an introduction that anything, hoping this group gathers some steam.

I'm a a developer/operations/security chap working for the UK Government Digital Service. I've got a particular interest in change under regulation and audit and have a fair bit of experience over the last couple of years of implementing (both technically and convincing auditors) continuous delivery. I'm pretty up on UK Government infosec, IS1/2 and have done a bit of standards work more recently. The following presentation I gave last year contains some relevant bits. 

Interested in who else is also interested in this topic?
1
Add a comment...

About this community

Vision: Define the authoritative guidance of how management and auditors should conduct audits where DevOps practices are in place, in support of accurate financial reporting, regulatory (e.g., SEC SOX-404, HIPAA, FedRamp, EU Model Clauses and the proposed SEC Reg-SCI regulations) or contractual obligations (e.g., PCI DSS, DOD DISA), or effective and efficient operations. By doing this, the Defensive Audit Toolkit will elevate the state of the management practice, defining how to understand risks to business objectives, correctly scope and the substantiate of effectiveness of controls, which reduces the costs of audits and increases effectiveness of audits. (This community can be access by http://bit.ly/DevOpsAudit)
http://bit.ly/DevOpsAudit

Gene Kim
owner

Discussion  - 
 
All -- +Jeff Gallimore +James D +Byron Miller and I have integrated all your comments up to CS1 and CS2.  We've also added CS3 to cover environments and infrastructure.  We're hoping to wrap up substantive updates to controls by Friday.

Please scrutinize CS3.  (All you puppet/chef/cfengine/docker/ansible fans will likely resonate with it.)

Thank you!

https://docs.google.com/a/itrevolution.net/document/d/1yGSBeKkqhavqk1I21f8UtJBWxDY_Q28nl3SEtcWNFPU/edit#
5
Gene Kim's profile photo
 
Okay, all our revisions are in, and all comments have been resolved.  +Jeff Gallimore would like to close the review process on Friday, Aug 8.  Please get your final comments in. 

Especially section CS3, which is new.
Add a comment...

Gene Kim
owner

Discussion  - 
1
Add a comment...
 
Through Growing Potential for Effective Enterprise Risk Management, the Requirement for Enterprise Governance, Risk & Compliance (GRC) Solutions Is on the Rise
Download Sample pDF of teh Report @ http://www.marketsandmarkets.com/pdfdownload.asp?id=1310
1
Add a comment...

Gene Kim
owner

Discussion  - 
 
It's here!  Early draft of the DevOps Audit Defense Toolkit is posted here -- we're 60% complete, I think, but we'd love comments and feedback.

https://docs.google.com/a/itrevolution.net/document/d/1yGSBeKkqhavqk1I21f8UtJBWxDY_Q28nl3SEtcWNFPU/edit#

+James D +Jeff Gallimore +Byron Miller 
8
1
Alex Wood's profile photo
Add a comment...
 
The video in this "AppSec in a DevOps World" post is well worth the 25 min investment. Lot's of great ideas supporting a DevOps Audit Defense Toolkit.
2
Add a comment...

James D

Discussion  - 
 
Are you listed as a public company?  
Are you being audited under attest standards (i.e., SOC or ISAE 3402)?  
Are you doing DevOps (initial, limited, full speed ahead)?  

Please comment below... I am working with Gene and the team here to begin solving this problem in our shared communities.
1
1
James D's profile photoRandy Tangco's profile photoGene Kim's profile photo
4 comments
 
+James D I am working with a colleague in the data center who manages the release management of products that runs in our boxes.  There are conversation with audit on this and based on what I have been provided, they get stuck on governance and discussion on this gets extended. 

Do you have a reference architecture for the deployment plan that I can read and hopefully use to build our own deployment plan? 
Add a comment...
 
""Users who can self-approve eg. developers"; #RunAsFastAsPossible" .. remarks from the twitter-infosec sphere.. Checky but important to know concern exists.  I also am seeing serious organizations allowing common end-users to self-escalate their privileges, so there is a culture of proof for #DevOps in the #Audit & #infosec sphere.  

Are you seeing the same cultural shift?
1
Mike Kavis's profile photoGene Kim's profile photo
2 comments
 
We've got "developers must never deploy their own code" captured as a common auditor objection/finding.  Any other auditor objections y'all want to put on the list?  Thanks!
Add a comment...

Dave Micko

Discussion  - 
 
Interested to see where this is going. What happens when I deliver features at a minimum once a week, yet still have to pass a SOC Type II audit annually? How can I educate my auditors that devops gives a clearer, more engineered approach to code management. Auditor developed tests for controls seems like they are stuck in the '80's...
1
Add a comment...
 
In the spirit of collecting useful content this presentation from James DeLuccia from Ernst & Young is excellent.
4
2
Larry Youngquist's profile photoPeter Haig's profile photo
Add a comment...