Agile at scale can get you to code very quickly, but then sometimes everything comes to a screeching halt. The biggest bottlenecks are often found after teams are done with the code. Dan James of Icon Agility Services joined Bob Payne on the Agile Toolkit Podcast to discuss Dan’s session at Lean+Agile DC 2018: Building a Lean Enterprise with DevOps. Dan and Bob explore “shifting left,” creating a pipeline of smooth handoffs, and decoupling release from deployment.
TRANSCRIPT
Bob Payne: [00:00:01] Hi, I’m your host Bob Payne I’m here at Lean+Agile D.C. 2018 and I’m here with Dan James from Icon Agility or is that Icon Agility Services.
Dan James: [00:00:13] It’s the whole name.
Bob Payne: [00:00:14] It’s the whole name? Okay great. And your talk is on DevOps Transformation, scaling and and other things.
Dan James: [00:00:24] Yeah, Extending the Lean Enterprise with DevOps.
Bob Payne: [00:00:27] Uh huh.What does that mean when you say that, Lean Enterprise?
Dan James: [00:00:31] Well we know that agile at scale can get you to code very quickly and it comes to a screeching halt because we have a wall of confusion – agile wants us to go fast. Business wants us to go fast. But the systems team wants stability right and reliability and security. And so you know our code comes to a screeching halt and may go into a black hole for weeks and months before it is finally releasable. And so what what we help enterprises do is work out the strategy and the tactics before we even talk about tools we get into the tactics and the strategy of creating a pipeline that smoothes out and leans out the handoffs yet to in order to get something delivered. And so we do a deep dive with our clients we go in and do a full technical assessment of of how they’re delivering value now. And we show them that their biggest bottlenecks are usually after the agile teams are done with the code and and help them get releasable a lot sooner.
Dan James: [00:01:35] And we give them strategies to protect their their product as they’re developing it by having you know green blue strategies you know delivery you know being able to separate or decouple release from deployment so we can go to production every day. Right. But it may not be releasable until the business decides we have accumulated enough real value share and then that becomes a business decision. So by separating it also gives us more time to smoke test and do canary releases and other things to ensure that what we have put out there is is sound before we release it to the public.
Bob Payne: [00:02:15] Yeah. So..feature Toggles those sorts of…
Dan James: [00:02:17] Exactly. And then we also teach the discipline of shifting left in the pipeline back to the teams. The responsibility for initial quality.
Bob Payne: [00:02:26] Right.
[00:02:27] So we don’t want to them to just throw their code over a wall and expect a testing team that had no input on the context of what they’re building right to think of all the possible edge cases to test this stuff. And so so we instill in our assessment we uncover all the the practices that need to be fixed before we automate anything and making sure that initial quality I mean if if if you think you can deploy quickly but you’re not unit testing your code then we have a big problem.
Bob Payne: [00:02:58] Yep.
Dan James: [00:02:58] You know the way that agile and scaled agile goes fast is by focusing on the quality.
Bob Payne: [00:03:04] Yeah.
Dan James: [00:03:04] And then we all go fast, And so that’s that’s the biggest thing.
Bob Payne: [00:03:08] Okay. For me I actually I actually believe the. So if you can’t get stuff out it doesn’t matter what your strategy is. I believe a lot of that the last mile work will allow us to shift lefter because ultimately I think one of the big problems that most organizations face in any sort of real agility they can get the wrong thing out faster but real business agility would use to use that for learning and it would have huge fundamental impacts on intake funding. You know lots lots of things that at least in the skilled agile framework they talk about but I don’t see many organizations actually pulling the trigger on that. There are certainly some in those sort of leading leading organizations will be the the sort of models that that people look at for a little while until it becomes more common.
Dan James: [00:04:14] Right.
Bob Payne: [00:04:16] A lot of people fail to understand the organizational possibility and the organizational impact of DevOps if done right.
Dan James: [00:04:26] Yeah and very often we go into an enterprise and we have to start with the real basics the fundamentals because they want to jump in to Agile because they’ve heard about it and it’s you know their competitors are already doing it. And so they’re at a tipping point. But they don’t even understand Lean. That’s where agile came from. And until they understand Lean and the waste that occurs in all the handoffs between each step in our in our operational value stream they don’t they don’t understand you know that agile alone isn’t going to get you it only gets half of I.T. fixed. But DevOps is the other half of I.T. and. And we’re going to show that in our in our speaking slot today we’re going to actually show here’s the here’s the the elemental chart of I.T. in general here all the departments that make a typical enterprise I.T. work only half of it is addressed by agile at scale.
Dan James: [00:05:21] So even a scale that only addresses half maybe 53 percent. Right.
Bob Payne: [00:05:25] Right.
Dan James: [00:05:25] It’s the other half that we’re trying to get which gets us time to market fixed it gets our products out the door gets the feedback from the customer that we are desperate to get. And it helps us learn and the whole principle of lean is is out learn your competition and then improve them. Share with you with what you learned. And if you can’t do that then you know we have to go all the way back to fundamentals. And so sometimes in our transformation engagements we have to we have to go back to the Stone Age of a year 75 years ago and talk about lean to get them to understand. You know it it still applies today and we can’t just say OK all our teams are gonna be scrum or all our teams are going to be Kanbun and expect it to solve all their problems and yet it only addresses half of the problems. You know what helps us get to code quick but it doesn’t do anything else. No it doesn’t. It doesn’t get the code out of the black hole you know before it gets released so. So we’re here to do that. We go into companies we do a deep dive a discovery an assessment of their of their DevOps side many of which are many of these companies are already doing agile at scale and doing it well but they’re still frustrated because nothing’s going out the door. And so we we helped uncover what most.
Bob Payne: [00:06:43] I might Argue that they’re not doing well if they’re.. if it’s not Going out the door.
Dan James: [00:06:48] That that’s true. And you know and you know Nirvana here is that the teams themselves have the power to release what they deliver or what they create.
Bob Payne: [00:06:56] Sure. Or to have an efficient way for that to that too certainly you know that may be an ideal to aspire to.
Dan James: [00:07:06] Sure. The Amazons of the world can do that.
Bob Payne: [00:07:08] Right. Well we are actually was just talking with Jeff Payne a little while ago which for us for the podcast listeners doesn’t make much of a difference. It’s on a different episode. But you know I think the potential for getting something out and getting it out. And you clearly articulated that those can be decoupled.
Dan James: [00:07:34] Yes. And and I think there’s a lot people are they are way too quick to say ooh that’s the next silver bullet teams team managed deployment. And for some organizations it is the perfect solution right. Lean Thinking looks at the entire ecosystem and is and tries to say what is the best solution for this organization. This team at this time with this technology and in so icy team managed deployment as a particular practice that may or may not be optimal in a given situation. So few people are looking in a lean way. They’re looking at other people’s recipes and that’s especially in size fits all right. I think this probably solves a ton of problems that we have with you know safety and risk profiles and and regulatory regulatory.
Bob Payne: [00:08:42] Yeah but you know looking at it as the next silver bullet I know I’m always caution even though I to work with organizations help transfer them towards this goal but only in the in in so far as we set a target we move along and steer and bright you know.
Dan James: [00:09:01] And I don’t know if it’s luck or curse that in the last three or four years most of my clients have been in the financial services industry which is highly regulated right. So so I banks and lenders and investment companies and so forth that that are under such regulatory burdens before they can really say anything to the public. And they’re under audit the threat of audit constantly and they’re scared of the audits that they use that as a wedge issue to prevent agility to prevent improving and and reducing the handoffs between the steps and getting value.
Bob Payne: [00:09:38] Even though you have much more closely auditable compliance.
Dan James: [00:09:41] Transparency, all that. Yes exactly.
Bob Payne: [00:09:45] You know, What I want the the developer to need root password to production to debug a production issue.
Dan James: [00:09:58] Right.
Bob Payne: [00:10:00] I think I would like the new way is a lot safer and more auditable and you know immutable immutable infrastructure networks are certainly those things provide a higher degree of safety audit ability than we’ve ever had before.
Dan James: [00:10:19] That’s right.
Bob Payne: [00:10:20] The problem is we need to ensure that teams are actually quite often that that the technique of audit and the things that you need audit need to change compliance are actually more compliance to the the spirit of those regulations than might have been when you had a big stack of documentation right which was only looked at when you need to practice for the auditor.
Dan James: [00:10:47] Yes. Yeah and we didn’t exactly exactly .. Static documents are obsolete that the day they’re published.
Bob Payne: [00:10:54] Right.
Dan James: [00:10:55] Yeah. So we found many of our clients have they’re so afraid of of the regulatory side.
Bob Payne: [00:11:01] Yep.
Dan James: [00:11:01] That that they’re just reluctant to release some of them might release once a year once or twice a year at the most. And they go through this long hardening period where they’re there waiting until the code was already written months ago before they even do a threat modeling penetration test against it. You know.
Bob Payne: [00:11:20] Thanks for making me snort. You know this hardening thing you know is there some sort of quantum stabilization of the bits in the silicone that I don’t understand.
Dan James: [00:11:33] And they don’t either probably.
Bob Payne: [00:11:34] Yeah I always think it’s just the bureaucratic way of leaving time for people to raise their hand and say we shouldn’t go.
Dan James: [00:11:41] Yeah and it comes down to fear. Right. You know that fear of release because they’ve been burned once or twice in the past when their technology wasn’t as good as it is today. And they get burned and now they’re they’re reluctant to release until they are just 100 percent of you know feeling secure. And so we show them methods of ensuring that the quality is there the threat modeling is already embedded that you know long before it gets even staging.
Bob Payne: [00:12:05] Right. And so with the proper strategy you can ensure your quality in smaller pieces and get it to staging or in production but not to release until you have enough business value that you can trust that what’s in production is clean and meets the security requirements meets compliance meets all those things. We’re just going to have to work in a more agile way to cut things up into smaller chunks
Dan James: [00:12:28] Make sure it’s tested upfront and early and often through the pipeline both both on the development machines and in the Coupée environments and in system integration environments.
Bob Payne: [00:12:39] Yep.
Dan James: [00:12:39] And that’s what it ensures multiple chances to smoke test this stuff and and make sure it is ready for release and we can be confident in. And then after they’ve seen a few frequent releases you know then their confidence builds as a as an organization. And the fear diminishes and they realize okay lean and agile and a scaled agile with with not necessarily the brand name Scaled Agile but agile at scale and DevOps is working. And then they they start feeling better. The auditors in many cases you sit down with the auditors face to face they’re going to audit on what you say you’re going to do great. So it’s good to show them that you’re going to do a new thing you have to sit down and talk to them and you know work. We’re going to do this a new way to please audit us on the new way not on the old way and the transformation becomes less fearful.
Bob Payne: [00:13:34] Right. It Can,.
Dan James: [00:13:35] It can.
Bob Payne: [00:13:35] Depending on your auditor.
Dan James: [00:13:36] Exactly so. So we’re in the business of helping that transformation and it’s a lot that a lot of the transformation is a mindset. It’s not so much the practices and the philosophy and the principles and all that even though we do we do preach that. It’s just it’s more of a mindset. And so we focus on the vertical structure above the teams to make sure they’re on board and they understand how to support this.
Dan James: [00:14:01] Yeah because a lot of agile failures come from the lack of support from above the teams you know so they the teams are constantly being injected in an artificial deadlines imposed and all the stuff that that kind of ruins Agile you know it ruins Scrum. You know okay, we’re right mid Sprint we’re going to be changing things. Well, The teams are frustrated and their productivity goes down because we haven’t properly trained management and that’s what scaling it helps us do it provides the training and the understanding above the teams.
Bob Payne: [00:14:29] Yep yeah at LitheSpeed we are focused a lot on leadership and transformational leadership as well. So most of our transformations start with with that sort of sponsorship but we’ve also started to try to create a path because management and leadership we’re not as well represented in early agile thinking mistakenly. I mean Sanjiv wrote the managing agile projects in 2005 and you know we started the agile leadership academy and then follow on. You know we created that. Now we’re starting to see things like Certified Agile leadeR and other programs for organizational leadership to really understand this which lean always had nice and is odd that that agile has taken as long to yes the teams and the ecosystem that the teams live in.
Bob Payne: [00:15:34] That’s right. And speaking of LitheSpeed we’re going to be there tomorrow and Friday my cohort from Icon Brian Aho and I are going to be at LitheSpeed.
Bob Payne: [00:15:42] Doing the DevOps.
Dan James: [00:15:42] And and training the SAFe version, the Scaled Agile version, of the DevOps course, which they accumulated from Icon. Mark Ricks for the last two years has been developing this and he and I and Brian have been teaching this now for the last nine months. But now we get to teach the SAFe version of it.
Dan James: [00:16:01] It’s now fully integrated into the Scaled Agile Framework they’ve added a few things to make it integrate a whole continuous exploration. So we turn DevOps into a scientific experiment.
Bob Payne: [00:16:10] right.
Dan James: [00:16:11] And small experiments just like Toyota did 75 years.
Bob Payne: [00:16:15] back to the future, i’m charging my flux capacitor even as we speak.
Dan James: [00:16:19] Exactly
Bob Payne: [00:16:21] Well thank you very much Dan great great having you here.
Dan James: [00:16:24] My pleasure.
Bob Payne: [00:16:25] Glad you’re able to speak at the conference and come to Agile DC as well if you’re interested. That’s the large local cross vendor conference.
Dan James: [00:16:37] And that’s in October right?
Bob Payne: [00:16:39] Yeah it is yeah. I’m the chair.
Dan James: [00:16:41] Yeah and I’ll be out here for the Scaled Agile Summit as well, the global summit in October so.
Bob Payne: [00:16:45] Great. Well we’ll see you at both those events and thanks.
Dan James: [00:16:48] awesome. Thanks Bob.
Bob Payne: [00:16:49] You’re welcome.
Listen to more Agile Toolkit Podcast.
