Actually, what a difference 6 weeks with the right team and the right focus makes. Last year, at around this time, the Healthcare.Gov site was in full meltdown, and all over the news media. Over the subsequent weeks and months, many of us spoke and wrote about what went wrong, how it could have been avoided, and how it could be fixed.
Providing rare insight into the rescue operation in the days that followed, is this landmark article from TIME Magazine. Here are some of the awesome agile things they did to rescue the beleaguered site:
1. Put Jeffrey Zients in charge, but, “the guy in the suit is never in control.”
A business executive, with a track record of delivering things, Zients was the solution to, No One in Charge. Though Zients was now in charge, it wasn’t quite as simple as ordering people around:
“Zients isn’t a techie himself. He’s a business executive, one of those people for whom control–achieved by lists, schedules, deadlines and incessant focus on his targeted data points–seems to be everything. So for him, this Apollo 13 moment must have been frustrating–because in situations like this the guy in the suit is never in control.”
“True, Zients had assembled a terrific team that had gelled perfectly. But his engineers could move only so fast. Though he had carte blanche to add resources, putting 10 people on a fix that would take one coder 10 days doesn’t turn it into a one-day project. Coding doesn’t work that way. “Jeff was a great leader, but there were limits,” says Dickerson. “He would ask us every day if we were going to make the deadline … He’d say how he had to report on how we were doing to the President. And I’d say till I was blue in the face, ‘We’re doing as much as we can as fast as we can, and we’re going to do that no matter what the deadline is.'”
2. Focused a core team under CTO Todd Park’s organization.
The site had been developed under the auspices of Health and Human Services. Puzzingly, President Obama’s CTO had not been involved in the development of the site. Mr. Park, as it turns out, is a huge fan and practitioners of lean and agile methodologies. In fact, Mr. Park was a keynote speaker at the 2012 Lean Startup conference, and shared his experience applying Lean Startup techniques within the public sector. So, they needed a solution to, Too Many Cooks?
From the TIME article:
“According to notes from a meeting in one of CMS’s three war rooms (yes, things were so uncoordinated that there were three), those assembled discussed the fact that “we heard that the capacity”–the number of possible simultaneous users–“was 100,000 people, and there are 150,000 people on it.” Yet five days later, White House chief technology officer Todd Park would tell USA Today that the capacity was 50,000 and that the website had collapsed because 250,000 people tried to use it at the same time. Park, a highly successful–but, for this job, disablingly mild-mannered–health care tech entrepreneur, had been kept out of the planning of the website. In fact, the site’s actual capacity at the time was “maybe a few thousand users,” according to a member of the team that later fixed it.”
So, at this late juncture, Mikey Dickerson, an Obama campaign veteran and site reliability engineer assembled a crack core team to fix the problem. Besides Jeff Zients and Todd Park, it included Ryan Panchadasaram, Jini Kim, Paul Smith and Andy Slavitt.
3. Worked in a rapid, but flexible manner to fix issues iteratively using a Daily Standup Meeting. Rules for this meeting:
“It was in a 4,000-sq.-ft. room rented by QSSI in a nondescript office park in Columbia, Md.–lined with giant Samsung TV monitors showing the various dashboard readings and graphs–that Barack Obama’s health care website was saved. What saved it were Mikey Dickerson’s stand-ups.
Rule 1: “The war room and the meetings are for solving problems. There are plenty of other venues where people devote their creative energies to shifting blame.”
Rule 2: “The ones who should be doing the talking are the people who know the most about an issue, not the ones with the highest rank. If anyone finds themselves sitting passively while managers and executives talk over them with less accurate information, we have gone off the rails, and I would like to know about it.” (Explained Dickerson later: “If you can get the managers out of the way, the engineers will want to solve things.”)
Rule 3: “We need to stay focused on the most urgent issues, like things that will hurt us in the next 24–48 hours.”
The stand-up culture–identify problem, solve problem, try again–was typical of the rescue squad’s ethic.”
All in all, we realize that this unfortunate saga could have avoided. So, the question that we need to ask is, why wasn’t Healthcare.Gov rolled out from the get go with agile and lean practices?
The TIME article has some strong thoughts on which we should collectively reflect:
Had the Obama team brought in its old campaign hands in the first place to run the launch, there would have been howls about cronyism. But one lesson of the fall and rise of HealthCare.gov has to be that the practice of awarding high-tech, high-stakes contracts to companies whose primary skill seems to be getting those contracts rather than delivering on them has to change. “It was only when they were desperate that they turned to us,” says Dickerson. “I have no history in government contracting and no future in it … I don’t wear a suit and tie … They have no use for someone who looks and dresses like me. Maybe this will be a lesson for them. Maybe that will change.”
As we head into next week, our @LitheSpeed team is preparing for a couple of days of fantastic talks (by executives from The Motley Fool, FireEye, AoL, Marriott International and the Millenium Challenge Corporation), lively panel discussions (Scaled Agile Framework and Lean Startup) and active networking opportunities; all at a great venue, the Center for Innovative Technology.
With the team moving in full flow, I thought I would disengage from the hustle and bustle of conference prep, and indulge in a bit of reflection.
The Agile DC Conference is my colleague Bob Payne‘s brainchild from over half a decade ago, and began with Bob and a small team of volunteers driving the event forward as a grand experiment under the umbrella of the non-profit Agile Philanthropy. Agile DC quickly filled an unmet need, and has become the Agile event of the year in the Washington DC metro area. These days Bob is supported by an all-volunteer organizing team, and the Agile DC conference sells out its 500 person capacity. Gallaudet University’s Kellogg Center provides the backdrop for this major event that now draws presenters and attendees from all over the country, and even internationally.
A couple of years ago, I noticed that while the Agile DC Conference does an awesome job of pulling together the agile community and others interested in agile methods, senior managers and executives weren’t that well represented or served by the event. We went through some discovery, and decided to pilot a new event, the Agile DC Summit last year. It turned out to be a resounding success, with executives from Mark Schwartz (CIO U.S Citizenship and Immigration Services) to Tracie Winbigler (COO of National Geographic Society) sharing their experience rolling out agile to transform their organizations.
We’ll be hosting the 2nd Annual Agile DC Executive Summit this October 20th at the Center for Innovative Technology in Herndon, VA. We’re super excited about the Summit this year and have a great lineup of speakers.
Who: Washington DC, Baltimore and Richmond area Executives and Senior Managers
What: The 2nd Annual Agile DC Executive Summit http://agiledcsummit.com
When: October 20, 2014
Where: The Center for Innovative Technology, http://cit.org
Thanks, and we hope to see you and your management colleagues in a couple of weeks at the Agile DC Executive Summit!
The growth of Agile in the public and private sectors has led to many cases of dysfunctional Agile – where organisations are mechanically going through the motions, but are failing to realise the benefits Agile has to offer.
At best, those companies that fall foul of the dysfunctional Agile trap are finding it frustratingly hard and costly to transition to Agile; at worst they’re losing significant market share.
And when this happens, Agile gets the blame.
In this video from Agile TV, experienced Agile Consultant Sanjiv Augustine touches on some of the root causes of dysfunctional Agile – and more importantly, how to avoid this common problem.
A key to success in engineering design is to “start with the end in mind”. Not, what does the end solution look like, or what will the end functionalities be, but ultimately what does the customer want from the experience, what is the experience outcome?
My first exposure to customer experience outcomes driving design lit up my imagination. It provided me a deeper level of “groking” what the purpose of engineers, (creative’s through developers) coming together to make something for someone to use, was all about. In 20 years of creating human centered design products and services, I have become convinced when you lead the think to make process you realize success far more often when it is predicated by experience outcomes. This is a paramount foundational principle of Lean UX.
To understand the nature of the customer experience outcomes (CXOs) just think of the long standing tradition of executives focusing on business outcomes to plan and deliver successfully for their company. Business outcomes are deeper than simply identifying greater valued business benefits that are derived from, say an outsourced product. The desired higher business value outcome would be faster to market, reduction in errors, and reduced costs for creating along with an increase in efficiencies. A recipe for success for what to go build, starts with the perfect blend of business drivers, customer drivers and technical drivers. Customer drivers = the desired customer experience outcomes.
I would like to coin a new identity to signify this focus, XFE or “Experience Focused Engineering“. If the core team can root all direction in creating experiences that customers love, we all win. This seems to simplify, or allow for focusing on the right things and not the process itself. Most of the 15 plus principles found in Lean UX can be summed up into making sure all your activity, actions and results deliver on the CXOs. Those tasks the customer/end-user wants or needs (even if they don’t know how to articulate it).
Breaking CXOs down
Customer experience outcomes start by identifying the desired results for what the user is wanting to accomplish. Then you clarify those by determining the metrics for success – or the evidence that met the acceptable standard. The core team now knows what to build and that they are building the right thing. This is also known as “Backward Planning”.
What is a Customer Experience Outcome or CXO?
The CXO is the “Enabler” + “Criteria”
The Enabler is what the product or service does that enables the customer to do what they value most.
The Criteria is how that enabler meets/succeeds at what is being done to meet the customer’s needs.
CXOs are often defined by the UX Researcher from the best measureable customer insights. They are simple statements that operationalize the top criteria our customers. We use them to judge the desirability, usefulness, and usability of our products/experiences.
- CXO’s define: 1) A critical task a product must do. 2) How well it must be done.
- A set of CXOs for a product is a plan for success: “If we do these things well, then our customers will love the experience.”
- As actionable insights, CXOs serve as data-driven building-blocks for the user journey. In turn, scenarios bring CXOs together into coherent, relatable end-to-end stories.
- CXOs optimize the user flow process,providing a greater end result with less work and more confidence.
- CXOs are measured within a scenario. They provide focus, efficiencies and flexible means of predicting product acceptance via a better understanding of how our products are usable, useful and desirable.
People love products that …
Let them do things that are most important to them [what],
in ways that exceed their expectations [how well].
Example from my Sinclair Digital initiatives…
How do we define a CXO to focus on?
Customer Experience Outcomes are derived from the first steps of “Design Thinking“.
- Empathy for what the existing problem/challenge is.
- Observe how customers deal with it and identify the key issues.
- Ideate the possible outcomes that are of value.
- Frame up the known desired outcomes and present to customers to validate.
What makes for a great CXO?
Specific – It is clear what the customer wants/needs and it is clearly understood by the core team.
Measurable – We know what success will be when the experience is delivered.
Attainable – It is able to be delivered in the user flow and by the technologies being implemented.
Ranked – Priorities as the most valued experiences to focus on
Tied to customer insight – Clear evidence is available for the insight.
Scoped – Defined to the minimal valued component valued by the customer.
Note; A CXO is not an “end-to-itself”, It should not be stated as the “solution”.
For example: Show me the news in a video player, is not appropriate for a customer experience outcome
Some key benefits of driving from CXOs:
- CXO’s clarify and drive the priorities for what to build.
- Serves as the “north star” for consistent alignment through development
- Insight for considering new products, services and features
- Inspire innovation for solutions with greater ROI
- Allow for 1 to 1 direct comparison to competitive experiences
- CXO’s provide the premise of our value proposition to end-users
Focusing on CXOs is a critical Lean UX principle that seems to be very lightly dealt with in the books and blogs that I have read. I find it difficult for my engineering teams to be able to deeply understand its significance. Let alone review best practices for realizing its full potential in our Experience Focused Engineering. To that end I am putting some training together to help deliver this transformational product design process at another level. If you know of any best practices around harnessing and mapping out quality experience based outcomes, please share them with me in the comments section.
I genuinely hope this three part series has provided some real-world insights into your adoption of Lean UX for your agile teams. As I continue to mature my experiences in applying these principles I will continue to share the knowledge, I look forward to learning from your shared insights as well.
A client of ours related an internal conflict at his organization regarding kanban and Scrum; which was better, and how do they interrelate? This post examines that issue. Also, check out our related interview with the author, Arlen Bankston.
Kanban is an interesting set of tools that can generate good data. For instance, measuring average cycle times for different service classes can help set expectations for work such as operational responses and defect resolution, in addition to new development delivery times that standard velocity tends to measure. If properly tooled, measurement is automatic, so this is useful, though not fundamentally different from velocity measures in Scrum. Team allocation is generally determined situationally in kanban, but given the costs inherent in large batch handoffs between skill sets, cross-fictional team structures such as those in Scrum are often still adopted as the most efficient and effective approach.
Nothing in kanban is contrary to or in conflict with core Scrum, and they’re frequently used together or hybridized. To whit:
- Many Scrum teams use kanban-style task boards; there are no rules about what exactly a board should entail in Scrum, just that work should be visualized.
- In a “Scrumban” style approach, a team might simply fill unused slots at Sprint Planning, and see how many get done on average in a Sprint to gauge Velocity.
- Policies around how much time to spend dealing with new development versus operations and so forth may be explicitly set via classes of service in kanban, or outlined in Working Agreements in Scrum.
- Extra stories are generally prepared in Scrum, then pulled if teams finish their committed work early, effectively creating continuous flow as in kanban.
With regards to the comment on continuous delivery, nothing in Scrum precludes it. Scrum prescribes a development cadence and synchronization opportunity with its time-boxes, but delivery could be (and should be) on demand. That Kanban is somehow better aligned with CD is a nuance that eludes my understanding, unless of course we are talking about the ideal of a “one-piece flow” – from concept to cash.
Scrum might seem overly prescriptive and dogmatic about its various ceremonies and practices but it was a breath of fresh air when first introduced, given where we were coming from. The surging interest in Kanban maybe fueled less by its solid lean underpinnings than by the notion that is it less prescriptive. That doesn’t necessarily make it easier or better in terms of adoption.
- The rigor and prescriptive nature of Scrum is actually helpful for new teams and those new to lean/Agile principles.
- Planned work that is inherently iterative in nature (cyclical and feedback-driven) seems better aligned with Scrum than Kanban, which is better suited to unplanned (doesn’t imply unknown), interrupt-driven items that follow a linear progression. For new product development, I would prefer Scrum.
- While Kanban doesn’t proscribe any specific planning meetings, it does imply and demand continual grooming of the backlog so that the team can always pull stories off the top of the backlog. If that is indeed the case, then some of the scheduled planning meetings are really context-driven and can be held only when needed, if needed at all. Scrum may frown upon it, but does it matter? In this case it’s eliminating waste.
- WIP limits constrain the amount of work in flight and are great for identifying bottlenecks and smoothing out the flow of value. The bane of time-bound Sprint with no explicit constraints on the number of items in flight work often results in last minute heroics in some teams and the risk of not reaching satisfying quality. Applying WIP limits on a Scrum board can actually tighten a sprint commitment.
- The ideal Kanban system is one of “single-piece flow” – one feature worked on by the team until done to satisfaction and shipped. This implies work being done in simultaneous phases which in turn demands high engineering discipline and an XP lifestyle. Short of that, teams would probably work on multiple features which could lengthen cycle time.
- The lack of a fixed time-box in Kanban may impede some of the predictability that iterations provide.
I could go on, but hopefully I’ve made my point. I see no reason to force either approach; let teams start where they’re most comfortable, and allow significant flexibility in technique and tool adoption while maintaining some reasonable standards around information capture and sharing. Teams will appreciate this levity and trust, and feel more ownership of their work, thus supporting better morale, quality and accountability. If you measure the impact of various things that teams try, the best approaches for various situations that are unique to your organization will simply become apparent over time.
Most Lean UX practitioners define documentation that follows, not leads, as the principle of “Getting out of the Deliverables Business“. The wording of this principle has caused a lot of confusion, and may create more damage than good when translated as “we don’t have to document anything”.
Agile development deliverables focus on the code and product subsets. UX Deliverables typically include myriad artifacts like thick BRDs (Business Requirements Documents), extensive wireframes, interaction flow diagrams, site maps, content inventories, taxonomies, contextual inquiry reports, trend analyses, and specification documents with full “redlines”. These have traditionally led the way, describing what developers are eventually to build.
I have found a simple refocus of the principle, where documentation follows the build, provides better understanding to teams adopting this principle. Documentation should follow and support each state of the Lean UX/Agile process, where measure-and-learn cycles guide pivots and happen rhythmically.
If you have a team of more than two people that have to get something done in tandem, they will need effective documentation; if you expect to build something that has more than a single simple button on it, you will need a minimum viable outline, or blueprint. In addition, if you bring other members into the core team later, or need to maintain what was built, you will need quality, albeit brief, documentation. For that matter, once I need to get more than 10 tasks done in a day, I need a list to help make sure I can track them.
Documentation is critical – but in an experimental, hypothesis driven approach it does not need to lead. Rhythmic measure-and-learn cycles can guide pivots, while minimal but sufficient documentation can keep the artifacts from holding inertia that makes change harder.
Lean UX brings a paradigm shift:
Leveraging Technology to Streamline the Documentation process:
Here is a practical tool I have discovered that can auto-generate a documented build direction when all the core team members participate. I was able to leverage new touch/pen large screen computers from Perceptive Pixel (currently owned by Microsoft) while working with a core team at Getty Images to define the interaction architecture. I found that a single OneNote page becomes incredibly powerful and time saving infinite whiteboard canvas.
This example of ideation (raw sketching documentation capture below) was done for a Getty brand mobile app experience. UX designers, researchers, and developers all came together in a room with a large touch screen in it and co-created the iterations for what the experience would work like. The digital sketching was being shared in real time to all the devices in the room. People were up at the board and working from the tablets on couches all from a single page within OneNote and in real-time. Beyond leading to huge time savings, by not wasting it on elegant and elaborate documenting exercises.
We enjoyed the benefit of the developers not getting lost in translation around the sketches. This was due to having given birth to the ideas in total collaboration alongside the UX designers. We were able to take our sketches right into prototyping.
Example of real world collaboration where documentation was auto generated from an interaction architecture sketching session:
The One Sheet Vision Casting Document:
Here is another real world example of a document supporting the team’s direction after the project kick-off. I use a single sheet to keep everyone on the same page (pun intended). I have found a single slide (minimum doc) method for complex MVPs can be extremely helpful since people actually read it, and can quickly refer to it during design reviews and demo builds. I have found this one sheet provides the stakeholders confidence in completely empowering the core team since everyone knows what we’re making and why, then makes together.
Example One Sheet - if you want to learn how to craft these for your team let me know.
The FUD Factor: (Fear Uncertainty and Doubt)
The paradigm shift to less documentation that supports rather than leads the product, can be unnerving to UX Designers and Researchers. These documents represent tangible milestones, and are used for their portfolios for future jobs; trophies to show how they contributed. Lean UX is a forcing function that gets the designers on the same engineering page as developers.
Now in order to fulfill your craft, you have to see the product or feature in action, used by customers. This supporting nature of the documentation becomes a true collaboration of team effort. One that you get to experience. Lean UX is about driving to quality, minimal, valued products with the least amount of waste.
Recruiters will have to dig deeper into interdependent and intertwined portfolios of actual experiences instead of the traditional artifacts of sketches, wireframes, drafts, comps and specs. It’s time to collect user feedback as your trophy of value to the team contribution. This principle empowers the UX designers and researchers to provide rapid build together insight and experiences without getting lost in the weeds of documentation.
I have found Lean UX principles to be revolutionary steps to transforming agile methodologies. It brings the end-to-end talent of the engineering pool together like no other tool. Lean UX provides the vision for harmony between the creatives, researchers and builders like never before.
Part 3 will focus on “Experience Outcomes“. I have become convinced that this is the most valuable principle of them all.
More shared learning coming.
I would like to highlight three key outcomes that applying Lean UX can bring to your agile teams. Let’s start with the base understanding of “Better Together”. These shared learnings come from my Lean UX workshop where I provide coaching for teams to practically and tactically adopt this new paradigm.
If you have been mastering the agile disciplines in your engineering practice, but have not connected your UX Designers and Researchers into the mix, you are missing some significant gains. You are leveraging the best of one practice while being dependent on incongruous processes that feed into your Agile projects, causing a less than robust agile practice. There is a clear and present need for the engineering process to be aligned end-to-end. By joining forces with all cross-functional teams, you can realize the ultimate goal of creating experiences that customers love.
Bringing the best practices of Cross Functional teams together.
Engineering spans “Thinking It” to “Making It”, yet most companies set up systems that tailor their processes to the different aspects of engineering. When builders participate in parallel with thinkers, they achieve contextual understanding together, which frees up the need to lead by documentation. No hand offs are needed, as they know what to build; they were an integral part in conceiving of the project. This dramatically reduces the need for things like redlines, specs and demo reviews that ordinarily take great effort throughout the process, as real-time communication is so much richer. In every case that I have experienced, life is better together when all participate together in an agile methodology with Lean UX principles and Design thinking applied throughout.
DIAGRAM: How it all comes together.
DIAGRAM 02: Working together from End-To-End in an Agile manner
This blending of two seemingly different disciplines, mindsets, even objectives of the same product only becomes strengthened as they participate in interdependence throughout every stage of the process. In the give and take collaborative creation of the idea through to the thing itself, each participant learns from the other SME (subject matter expert). And each gains deeper understanding of the why’s involved. Firsthand experience by both types of talent in the engineering process in the thinking to the making removes the need for documentation to lead and dramatically reduces the “lost in translation” phenomena. I have found that when the team plays together in each other’s worlds – the divisive behaviors end, the comradery and buy-in increases and a joint passion for the projects and product enables the teams to deliver a much higher quality experience. Shifting to this interdependent “better together” model is the beginning for teams to transform beyond normal performing.
Part 2 will cover the Lean UX principle of “Getting out of the Deliverables Business” and hopefully add a little more clarity around what that actually looks like.
Part 3 will focus on “Experience Outcomes“. I have become convinced that this is the most valuable principle of them all.
So stay tuned!
Blog post submitted by Bob Payne, VP of Enterprise Consulting, reflects on the legacy of Jim Weirich.
This past February the Agile and Ruby communities lost a very influential member. Jim Weirich died on February 19th. Jim was involved in the creation of many things that the ruby community now takes for granted; Rake, RubyGems. Github honored him with a special message for his last commit on the [Wyricki project].
Unfortunately I only met Jim one time. However, I do have a podcast as a memento of that meeting. Back in 2007 I spoke with Jim at the RailsEdge and wrote the following intro to my podcast. You can listen to the podcast at AgileToolKit.
Here is a short excerpt from the interview:
“I spoke with Jim after his Rake Talk at the RailsEdge. If this is not your favorite build tool now it will be. Jim is the creator of Rake and the RubyGems package management for Ruby. These two tools have made my life a lot easier and if you have not used them, then you are probably not using. Jim talks about using Rake and db migrations with a recent java project.
I also spoke with Jim about ruby test driven development. It was great to see someone do tiny step TDD in a presentation that was not an eXtreme Programming demonstration.
Jim is the first of my guests to say “You’re the AgileToolkit Guy…I listen to the podcast!” or something to that effect. Thanks for the interview and for listening to my podcast Jim.”
There is now a scholarship fund set up in Jim’s name for computer science students. With this fund we can help carry on the legacy of teaching that Jim showed in his lifetime. More information about the fund can be found here:
This is the third in a series of posts about the ongoing Healthcare.Gov saga. In my first post, I suggested that the project had simply too many cooks in the kitchen to be successful. In my second post, I looked at the absence of a “thin slice.” In this post, let’s take a look at something glaringly obvious: no-one in charge.
Most government projects still follow the waterfall software development method. Waterfall projects are typically managed using a common-and-control style of management. As Joel Sposky points out, the command-and-control method, while it works well with 18-year old soldiers, is not the right choice for knowledge workers like highly-skilled software developers.
In a command-and-control system, unlike lean or agile systems that maintain a “responsibility based” organizational structure, some has to be in charge for the system to work.
Despite having having a very large team with over 50 contractors, no one was in charge until President Obama appointed Jeff Zients to take over the failed effort and set things right. So, the government blames the contractors. The contractors in turn blame the government. The contractors also blame each other.
Mr. Zients’ first public move? He elevates contractor QSSI to the lead, and puts them in charge of overseeing the effort.