Scrum Institute, Scrum Framework Episode #7 has been proudly brought to you by International Scrum Institute, https://www.scrum-institute.org
You can also listen Scrum Institute’s Podcast from Apple, Spotify, Castbox and Google Play.
Listen to Scrum Institute Podcast on Apple
Listen to Scrum Institute Podcast on Google Play
Listen to Scrum Institute Podcast on Spotify
Listen to Scrum Institute Podcast on Listen Notes
Listen to Scrum Institute Podcast on Castbox
What Can Cause Chaos And Frustration Before The Scrum Framework? This Might Surprise You!
To better understand the impact of the scrum framework to our software engineering practices and businesses, it makes sense to have a look at a day in the life (or a software project in life).
Therefore, I would love to briefly talk about a software project from the past before we adopted the scrum development and software delivery framework in our organizations.
A few days before I wrote these lines, we had lunch with one of my ex-colleagues with whom we used to work together almost 20 years ago.
This gentleman, Marcus has got his scrum master certification and scrum product owner certification from International Scrum Institute™. He currently works as a scrum master for one of the leading software houses in the agile project management software domain.
As a scrum master, Marcus is now in charge of operating an agile scrum team whose scrum team members located in geographically distributed locations around the globe.
During our lunch, Marcus admitted that there are a lot of typical challenges with distributed agile scrum teams. Some of the problems he specifically mentioned related to his software project configuration are:
- Differences in working styles among scrum team members,
- Timezone differences,
- Cultural misfits, and
- Language constraints.
Despite these difficulties, Marcus still added that running a software project with the agile scrum process is more fun, productive, and enriching than how we used to work 20 years ago. Compared to days when we used to work without scrum software development and scrum software delivery processes.
Marcus’ statement was indeed a big testimonial for the credit of the scrum framework from a very accomplished and experienced manager, scrum master, and product owner.
Thank you, Marcus!
Then we explained to him one of our past software projects before we used to meet with the scrum framework. I’m sure that many scrum masters would resemble this experience to their previous projects before they’ve gotten their scrum master certifications.
Back in the late 1990s, we were part of a software engineering group to build a smart card-based public key infrastructure. Smart cards securely protected private keys of infrastructure members, associated public keys and their wrapper certifications were openly distributed (as the name public implies).
Back in the day, this was by itself a relatively complex IT project that required multiple interdependent hardware engineering and software engineering teams. We had to do massive amounts of research and development (R&D) to build a fully functional hardware and software system.
Remember these are days before we had the minimum viable product (MVP) concept to experiment, create, learn, and experiment again.
Without scrum to create such a sophisticated infrastructure that constituted numerous hardware and software elements was a real challenge.
Here are three significant setbacks we used to have without any scrum masters and anyone who possesses a scrum master certification in our teams.
Had To Plan The Entire Project Before Understanding The Project? This Might Surprise You!
Without scrum, our teams had built and delivered entirely wrong software and hardware products that did not fulfill demands from our client.
We had times in our professional lives when some third party companies had imposed how we supposed to build our software products or software services.
Capability Maturity Model (CMM), ISO 9001:2008 and other derivates attempted to help our companies to ensure we build our correct software in correct ways.
How successful they used to be is not part of this book. This book was meant to focus on the scrum process and meri ts of the scrum framework rather than criticizing almost extinct procedures.
However, I have to add that these process improvement frameworks before the scrum software engineering framework recommended a phased approach. They advised a phased software engineering approach which we called the Waterfall Software Engineering Model.
With the Waterfall Model, each software project was supposed to start with requirement analysis, where we aimed to understand what our client needed and wanted.
Then we designed these requirements, we implemented them, we tested (verified) them, and we maintained them in our software production environments. Finally, we reached to end of the software engineering lifecycle.
Nonetheless, the reality didn’t play out like that!
The Waterfall Methodology vs The Scrum Framework
Phases in the Classical Waterfall Software Development Model
The adverse effects of unforeseen delays happened during a particular phase of the Waterfall Software Engineering Model were inevitable to the following software engineering phases.
Studies have shown that in over 80% of the investigated and failed software projects, the usage of the Waterfall Methodology was one of the critical factors of failure. But why?
As shown on the left side, when deploying the Waterfall Methodology, there is a strict sequential chain of the different project phases. A previous phase has to be completed before starting the next phase. Going back is, in most cases, painful, costly, frustrating to the team, and time-consuming.
The project timeline is planned at the start. A releasable product is delivered only at the end of the project timeline. If one phase is delayed, all other phases are delayed too.
To avoid this, project managers of the Waterfall Methodology usually try to anticipate all possibilities beforehand. That means that in one of the earliest phases of the project, they try to define all requirements as fine-grained and complete as possible. However, requirement definition in an initial stage of a project is often complicated, and therefore many requirements change (or should change) throughout the project.
Studies have shown that in more extensive and complex projects, about 60% of the initial requirements do change throughout the course of projects. Other requirements are implemented as define, but some of them are not really needed by the customer. So those implementations consume time and money that could have been better used to implement functionality with a higher added value for its clients.
The separation into different project phases forces project managers to estimate each phase separately. The problem is that most of these phases usually are not separate. They are working together and in parallel.
For instance, no reasonable human-being can assume that the development phase finished before the testing phase started. And yet, this is precisely and unfortunately how the Waterfall Methodology used to work.
The Waterfall Methodology for developing software can be used for implementing small and straightforward projects. But for bigger and more complex projects, this approach is highly risky, if not insane. It’s often costlier and always less efficient than Scrum software development and delivery framework.
This was the life before the Scrum framework. Sending our software back and forth between various teams, without the guidance of professionals with the Scrum skills, made our work bureaucratic, complex and unproductive.
Finally, it wasn’t only the product which suffered, but also employee morale and commitment to our organizational mission have wholly disrupted.
Overcoming A Lack Of Commitment, Change Management & Team Work! This Might Surprise You!
The most significant weakness of process improvement frameworks used before Scrum was that: They mainly focused on self-serving organizational demands of leadership.
Some of these demands are monitoring, compliance, and predictability. There was no focus on serving clients well and increasing employee morale at all.
Thus members of software management teams and various other internal and external stakeholders attempted to have a fixed deadline for software delivery projects and easily monitor the progress of software engineering phases.
They penalized their people if something was outside the planned track, and they hoped to fix emerging issues before the scheduled date of project completion.
Furthermore, independent silos realized entirely separated software engineering phases. As an example, the development team was completely independent of a software testing (verification) team. Most people who supposed to work for the same business mission didn’t even know each other by their names.
Have you got a guess about the reason for this silo-mentality in our organizations rather than focus on business missions and professional (business) maturity of employees?
The reason is simply the matrix management . Matrix management is an organizational management and employee structure, and it has been in our businesses since the 1970s. At first glance, the differentiating idea behind a matrix organization or matrix management seems to be smart.
The Leadership creates an organizational structure by bringing together employees with similar kinds of functional and technical skill-sets into the same or at best neighbor silos.
The Waterfall Project Delivery Model in a Matrix Organization
Back in times, it was quite popular to see the so called “Center of Competences” in our companies where each center of competence represented an independent and autonomous silo.
One silo for C++ developers, another silo for database administrators, and another entirely separate quality assurance silo in oversees and it goes on and on. Go and figure!
The biggest challenge with the matrix organizational structure was that: To deliver a software project without the scrum framework and scrum masters, project managers had to borrow employees from silos temporarily.
These employees did not even physically position with their project teams, but they still located in the rooms of their particular center of competences.
Up upon completion of projects, these temporary project teams dissolved and project participants moved on their next assignments to serve for other projects.
Therefore, the targeted business values of these ongoing software projects have never been the utmost priority for these independent silos.
They tend to see their work as checkboxes they ticked for one project over here and another project over there.
Leadership and matrix organizational model didn’t teach them how professionals should commit their business to improve the bottom line, including sales, revenue, and profit.
A McKinsey Quarterly article written by McKinsey & Company has also clearly illustrated this illusion of cost optimization beyond the matrix organization .
Gartner has estimated that organizations worldwide have been yearly spending 600 billion USD to recover their IT systems from non-scheduled maintenance work and defects.
Now let’s take a short moment to visualize how the change management and impediment handling of software projects played out. How t hey pl ayed out i n a proj ect configuration with the waterfall model, with the matrix organization, and without the scrum process.
Yes. You’re right.
Management and employees treated change management, impediment, and error handling as if they’re ill exceptions which shouldn’t have happened in the first place.
Therefore, changes in a software project have been the synonym of delays. They usually created a domino effect of cascading delays. Teams required someone to blame and finger point for defects and impediments.
Last, but not least, because silos did not have a mechanism in place to process, fix, and learn from their errors, they kept on repeating the same mistakes.
Furthermore, they kept on augmenting the amount of technical debt while they passively attempted to deal with their problems.
Why Should Democratic Decisions Not Be Overruled? This Might Surprise You!
Steve Jobs once said:
“It doesn’t make sense to hire smart people and tell them what to do. We hire smart people, so they can tell us what to do.”
However, this is precisely opposite of how most of the mainstream leadership used to operate to make decisions before the scrum era.
Before we had the scrum process in our organizations, autocratic decisions from leaders overruled the combined intelligence of their teams.
They invalidated the democratic decision making ability of groups who were in charge of doing the real works which spanned the entire software engineering lifecycle from the conception of software to its operations.
The remoter a decision was shifted away from work centers (teams) it impacted, the more default it was to give a correct mission-critical decision.
The judgments from leaders used to be usually impulsive, not thoroughly thought-out, mostly late and tentative, but sometimes even too early.
These autocratic decisions imposed from the top made employees feel undervalued. They entirely hindered the ability of their organizations to come up with creative and innovative solutions to handle competitive business and software-related challenges.
Furthermore, they discouraged software engineering teams from giving their inputs at the times when they’re asked to contribute decisions.
It was a brief overview of how we used to develop and deliver our software services and service products before we adopted the scrum framework in our organizations.
Now let’s have a look at how we sorted out these chaos and frustration elements with the help of the scrum process.
What Makes The Scrum Framework Succeed? This Might Surprise You!
The Scrum Framework changes the classic triangle of project management.
Organizations do no longer need to sacrifice one of the time, budget, or quality. The new triangle is now emerging between the budget, time, and functionality. And none of these project success elements have to be endangered.
Triangle of Project Management
According to the Scrum framework, quality is no longer optional. To deliver what clients are paying for to flourish their businesses, the Scrum Teams strive to provide the best possible software they’re jointly able to build.
In the Scrum framework, the factors which define when a feature is complete and when it meets the required quality standards are set by Definition of Done (DoD). DoDs specify the expected outcome in terms of functional and non-functional requirements, design, coding, unit testing, end-user validations, documentation, and so on. DoDs are defined in the levels of both user stories and tasks. DoDs of user stories focus on functional and non-functional client requirements, whereas DoDs of tasks focus on the desired working activities from the Scrum Team members.
The Scrum Team is not allowed to close the user stories, and obviously, the tasks that do not fulfil their DoDs. Scrum Product Owner and the Scrum Team define user stories and their tasks throughout the course of the Scrum software engineering process incrementally.
This incremental development allows the team to remain adaptive and adjust their next best actions in a controlled manner without the additional costs and risks of jeopardizing large chunks of previous work.
The Scrum Team builds a potentially shippable software product increment until the end of each Sprint. The team demonstrates and discusses these increments with the Scrum Product Owner and client stakeholders to get and incorporate their feedback towards the next steps of their project.
This flexibility applies to not only software delivery but also the operational processes. So, the Scrum Framework allows the optimization of the use of resources (human, time, budget, material) and the minimization of wastes.
Studies have shown that Scrum has the following positive effects in practice:
- More frequent code deployments,
- Faster lead time from committing to deploying code,
- Faster mean time to recover from downtime,
- Lower change failure rate,
- Better product quality,
- Reduced or identical costs compared to pre Scrum deployment,
- Improved productivity and throughput,
- Improved code and operational reliability,
- Enhanced organizational performance and client satisfaction,
- Improved market penetration, market share, and profitability of organizations,
- Improved market capitalization growth,
- Improved motivation of employees.
Introducing and adopting the Scrum Framework is non-trivial. And yet, the adaptive and iterative approach of the Scrum Framework handles this initial burden, and it copes with ever-changing client and business requirements better.
Thus, the Scrum Framework is, in most cases, a better alternative to the classical software engineering methodologies.