A little history
You might remember how we made software a decade ago – "we" being the entire software industry, Microsoft included. By today's standards, slow ... as ... molasses.
We released products on two to three year cycles. It wasn't unheard of to spend six months to a year planning, a year or more coding, and then months packaging everything up for delivery. It worked in the software generation in which it was born, but today's environment requires a different approach. Today we buy software daily—often with just the touch of a finger on the device we're carrying with us. Market opportunities come and go more quickly, and customers demand increasingly faster turnaround to meet their needs.
Everything we do around development — collaboration, testing, and customer feedback — has changed.
Here in the Developer Division at Microsoft, we needed to adjust our business to respond to these changes. The old way was no longer working. Agile methods and practices offered some hope, but how could we bring agile to our entire division? We had hundreds of teams across thousands of engineers. You see, we've had agile "pockets" for years throughout Microsoft, but nobody really knew whether agile could be rolled out successfully at this scale.
We knew we needed to transform our business. The time had come to change... or be left behind. This story is about how the Developer Division at Microsoft made an agile transformation that scaled to support business needs.
Waterfall vs. Agile
Today, we think about how fast we can translate an idea into reality, and get it into customers' hands.
We started by re-thinking how we work. Our goal was to build software more incrementally, deliver it to customers faster, and get feedback and input that helps make the end result even better. Building software has always been a creative process and we needed to find a way to let teams "create" again. Teams needed more autonomy to think outside the box. They needed to be empowered to make decisions without being held back by the process. We needed to fundamentally change the way we work.
Visual Studio Transition
We needed to work more incrementally, deliver to customers faster, and let feedback make the product better.
The Agile Shift
We didn't decide we were going to be agile starting tomorrow. There was gradual buy-in with teams and leadership.
We started by taking our senior executives in the division through a bit of agile training. We did a project where everyone had to design and build a game. At the end of every hour, each participant had to play their game and demonstrate their progress. This repeated until the games were "done." It was a real eye-opener for the management team to see agile in practice. They weren't reading about it or hearing about it from a colleague. They were doing it, and seeing the impact first hand.
A new cadence
As an organization, we knew right away that if we were going to build in smaller increments, we needed to think in smaller increments. From that moment, our entire schedule shifted from multi-month milestones down to continuous three-week iterations. I'll never forget Sprint 1. It was our first three-week iteration, and I don't think a single team was successful. It was a train wreck. However, we took the learning from that sprint, made a few adjustments, and got started on Sprint 2. At the writing of this article, we're halfway through Sprint 64; we've been using this approach ever since.
The new normal
A typical sprint looks as you might expect. Day 1 is sprint planning, where teams discuss what's next on their backlog. Teams gather to talk about each story and make sure they understand all the requirements. This can take some back and forth, but the goal is to build a plan for the 3-week sprint in a single day. The entire team works together on this.
We moved out of our individual offices, and put teams together into the same room.
Next, we execute. Code is written and tested daily. In our daily stand-ups, teams gather to talk about what's been accomplished, what's next, and if anybody's blocked. The key here is that we're not just writing code during a sprint. We're writing code, testing that code, and making sure the code is "shippable." At the end of the sprint, each team holds a retrospective where they talk about what's working, and what could use some adjusting—reviewing and assessing so that they're constantly improving.
Prior to this approach, we separated coding, testing, and upgrade tasks. Coding might happen months before testing. More than anything, the old way created a technical debt problem. The longer we wrote code in isolation, the more debt we created. The bug curve grew and grew. One of our new principles is that we don't carry technical debt forward. We pay it as we go every sprint allowing us to remain in a "shippable" state at all times.
And the end of a sprint, each team sends out an email listing all the work completed together with a quick demo video showing changes to the product. The video helps us celebrate the work and keeps everyone abreast of progress.
The New Normal
We need to make sure we're hearing our customers, and learning from them as we're building the software.
The process worked well for teams, but we needed a new way to communicate with our leadership team as well. We need to stay connected and informed and our answer was something we call "Team Chats." Every three sprints, we invite teams to sit down with leadership to answer three basic questions: Are there any issues in the way, any impediments? What's your technical debt situation? What's next on your three-sprint roadmap? This approach allowed us to communicate across teams and make adjustments as the software came to life. It wasn't about following a plan laid out eighteen months earlier; instead, it was about learning as we built the software. Adapting it as we went.
We'll talk about what's working and what's not working... the idea is continuous improvement.
This new way of working is very cyclical. We write something. We give it to the customer. We ask them what they think. We write something. We give it to the customer. We ask them what they think. It's a continuous and steady buildup toward the final product, instead of a big push all at once. The software is developed based on an ever-evolving understanding of what the customer wants. This helps us get to the end result faster and helps us get it right—thus eliminating waste. For the team, it's very rewarding.
A new mindset
We also adopted a mindset common in the agile community—"fail fast." A Waterfall approach to software development gives you only one shot at getting things right. If you try and fail, there's very little time to make adjustments. Agile is different. It's expected that you'll get some things wrong, but it's also expected that you'll learn from your failure... and that the learning will lead you to a better end result. Teams can run an experiment—try something for a sprint. See how it goes. If it doesn't work, well, it was just a sprint.
We need to know that what we're building scales for software teams around the world.
All these changes have helped us deliver a better product, but they've also directly contributed to a boost in morale. Employee satisfaction is a success measure. In the old world, a team might have to wait a year or more before a customer exercised the code they had written. With a more frequent delivery cadence, it's now just weeks between checking something in and having it actually show up in production. That's rewarding. Once we switched to this rapid cadence, teams never wanted to go back.
Eating our own dog food
We also ensure that we're using the product we're building. We call this "dog-fooding." We need to walk-the-walk and talk-the-talk. We need to know that whatever we're building works for software teams of different sizes and scales and shapes and complexities around the world. That means making the tools highly visible and flexible. We use Visual Studio, Team Foundation Server, and Visual Studio Online every day to run our business. If we can't use them to run our own business, how can we expect our customers to?
In June of 2013 we took an even bigger step on our agile journey by changing our physical environment to support the changes we were making to our process. We moved the majority of our division into a building completely built around team rooms. This new building didn't look like the Microsoft buildings most of us were used to. Individual offices were no more, replaced by "team rooms" designed for a cross-functional team.
In a traditional building, each team member has a private office. In some cases, the development team, program management team, and test team for the same project are all in different buildings. This isn't ideal, so Microsoft completely redesigned the building to reflect the agile mindset. We wanted teams of 10 to 15 people to be physically together, working across disciplines—program management, development and test—all in one space. Standups happen in the room instead of having to gather in a hallway or a pre-scheduled conference room. Desks are even on wheels so teams can re-organize as needed without overhead. The space adapts to fit the team.
Each team room is also fitted with a conference room and focus rooms for private conversations. It took some time for everyone to acclimate to this new environment, but we've found that the physical changes contribute to helping us build better software. Teams love it.
Before you leap
It's important to understand that our agile transformation has been a journey, not a jump. Many organizations want to be agile tomorrow, but it doesn't happen overnight. Below are a few "rules of the road" we developed along our journey.
Your software development efforts have to aid your business. That's why they are there.
Take it slow. We didn't decide one day to flip a switch and have everyone start doing things differently. We literally took three years and went team by team, onboarding them when it made sense to do so.
Business needs. In a large organization, whether it's Developer Division at Microsoft or any Fortune 500 company, software development exists to aid the business. That's its purpose. Don't try to be agile just to be agile. If it's not aligned with a business need, it's not going to work.
Persevere. We've found that agile practices are the best way to achieve maximum ROI. Don't give up after your first failure. Let that failure inform your next steps, and then get back to it. Even if your first few sprints are disasters, hang in there, because it is a big shift.
Listen. Ultimately, customers are the ones who will tell you if you're doing it right. You might have a brilliant process, but if you can't sell what you're producing, then you've got a bigger problem. Listen to what your customers are saying. What we discovered on our journey is that the product we're producing is better. Our customers tell us that, and we can see it.