Situation: Risk Outweighs Reward in Many Plan-Driven Software Development Cycles
As software becomes more expensive and risky to develop, some companies are looking beyond traditional plan-driven and defined approaches. Such approaches attempt to mitigate risk and minimize waste with a structured development cycle. Ironically though, this very structure can contribute to inflexibility and obsolescent requirements, without creating a sense of success or failure until it’s far too late to make significant code adjustments.
Traditional planning asks developers and shareholders to make critical choices regarding requirements, personnel, goals, and budget with imperfect information. Development teams may try to make major design and development decisions at project outset, with many of these decisions involving what amounts to educated guesswork. In a worst-case scenario, companies can discover their initial requirements or budget to be inadequate after investing thousands of hours and millions of dollars, creating the unpleasant and costly task of scrambling to salvage their investment.
Problem: The Perception of Agile Development is One of Irresponsibility
Agile software developers abandon stringent structure for collaboration, rapid response to change, and interpersonal interaction. Plans are loose; individuals are paramount. Its proponents claim increased team productivity, rapidly usable code, and adaptability. Because it delivers code often and early, the team can constantly re-evaluate and respond to unexpected requirements and derail ones prove to be minor. Agile’s detractors denounce it as careless, planless, and without the means to adhere to budget and time constraints like a defined, plan-driven approach.
On an abstract level, most project managers will choose adaptability over rigidity. These managers will not, however, agree to an agile strategy that forsakes any form of architecture, delivery plan, or unit testing. While this myth is understandable, it is nonetheless misguided.
Solution: Scrum Lends Structure and Discipline to Agile Methodology
While Agile developers favor collaboration and flexibility, successful ones do not abandon planning and architecture. At its heart, Scrum strives to resolve the conflict between planning and flexibility. It seeks to leverage Agile’s strengths while maintaining a reasonable level of control and predictability.
“Scrum concedes that successful Agile development requires discipline and organization.”
Scrum concedes that successful Agile development requires discipline and organization. Scrum assigns defined roles, clear goals, and uses daily meetings to constantly assess and reassess the project. In the end, Scrum delivers the right product within a reasonable budget and in a timely manner.
Magenic’s own brand of Architected Scrum goes a step further by conceding that shareholders want hard targets and a definite cost at outset. This formal step, called “Envisioning,” allows Magenic to address high-level architecture decisions during the initial stages of the development cycle.
Result: Architected Scrum Eases Shareholder’s Fears While Maximizing Profit
Architected Scrum augments Agile’s strategy of iterative development and collaboration with clearly defined goals and a moderate level of architecture. Applied properly, the team will see improvements to revenue, speed to market, quality, cost control, customer satisfaction, and other elements.
Most shareholders prefer a clear determination of goals, budget, and deadlines, but the Agile developer believes this process is flawed at the core because it attempts to designate structure with only partial understanding of the path ahead. Agile teams enjoy increased productivity, adaptability to changing requirements, a more rewarding work environment, and usable code at early stages in the cycle. Agile methodology promises much, all in the name of a better product and more profit.
- Establish flexible, adaptive, yielding team communication
- Increase productivity
- Demonstrate usable code throughout the project
- Adapt to changing requirements when necessary
- Buy-in and improved adoption of new systems by users via involvement in the Agile process
“Architecture in Agile Projects,”
Presentation in Orebo, Sweden
Agile teams identify the most important features before beginning work and build outward. This helps to streamline the entire process by eliminating bloat and unneeded features. Agile adopts iterative development, focusing on small goals to deliver usable code throughout the project. Requirements are gathered over time rather than in bulk at a premature juncture. This methodology expects and understands that business needs evolve over the duration of a project. The Agile process enables the team to address the customer’s wants, needs, and business objectives. If a previously unforeseen but critical requirement arises, then the team makes it a priority. Conversely, if a requirement turns out to be less significant, then it is removed or diminished.
The possible danger lurks in Agile’s dependence on collaboration and communication. Agile necessitates that someone take charge, assign goals, create deadlines, and keep team members on track and accountable. That’s where Scrum enters the picture.
Scrum: Slow and Steady Wins the Race
Advantages of Iterative Development
- Increased revenue
- Increased speed to market
- Better quality
- More visibility
- Improved risk mitigation and cost control
- More flexibility
- Maximum customer satisfaction
Scrum keeps an Agile team working cohesively with frequent team meetings during an iterative development process. This process creates an environment of pervasive self-reflection and assessment of goals, methodology, and code. In short, Scrum rejects the traditional “Big Design Up Front (BDUF) vs. No Design Up Front” argument in favor of “Enough Design Up Front” (EDUF). In practice, however, design never stops. The Agile team takes time at the beginning of each sprint to reassess.
Scrum divides the development cycle into short time cycles called sprints, each of which is typically two to four weeks in duration. Each sprint focuses on producing demonstrable work ready to hand to a customer or present to a shareholder, and each sprint ends with a review and planning for the next sprint.
The rewards for properly applied Scrum are substantial, but none of them can be realized without discipline and organization. The EDUF philosophy requires frequent meetings with associative costs, the maturity and dedication of all participants, and high levels of communication.
Profound, meaningful communication requires regular interaction and efficient means of collaboration. Scrum typically adopts six types of meetings to facilitate coordinated collaboration throughout the iterative cycle, though these do not necessarily need to be face-to-face. Once again, the meetings serve to foster constant self-reflection and assessment of the team’s goals, methodology, and code. At its heart, Scrum uses this process as a means to avoid burnout and favors a consistent pace rather than pure speed of execution.
As keeper of the process, the Scrum Master removes both local and global impediments to progress and helps team members self-motivate.
The Daily Scrum Meeting serves as the daily project meeting and asks the following questions of each team member:
- What have you done since yesterday?
- What are you planning to do today?
- What are impediments and stumbling blocks for the day?
Storytime (also known as Backlog Grooming) examines the existing backlog and refines the acceptance criteria for individual stories. It divides monolithic design documentation into smaller, more specific story-relevant documents to enhance accountability and progress tracking. In addition, Storytime allows the team to connect to supporting documentation to related stories, database documentation, and performance requirements documentation.
The Scrum of Scrums occurs after intervals tailored to specific project and team needs, though only one representative from each team attends. Some companies employ it daily, though Magenic holds the Scrum of Scrums weekly to reduce overhead costs. It asks questions such as:
Removing Impediments—Best Practices
- Make impediments transparent and known to all.
- Be proactive and search for them.
- Limit them: they must be solved or trashed with 24 hours.
- Differentiate between global and local impediments
- Global slow down the team
- Local ones block progress on stories
- Help the team solve impediments so they can solve them on their own in the future.
- What has your team done since last we met?
- What will your team do before we meet again?
- Is anything slowing your team down?
- Are you about to slow another team down?
The Sprint Review Meeting takes place at the end of every sprint. Its purpose is to review all work – finished and unfinished – and present completed work to the shareholders. It never presents incomplete work.
Finally, the Sprint Retrospective also occurs at the end of every sprint. During the meeting, team members reflect on the past sprint in order to make continuous improvements to the process by asking “What went well?” and “What could be improved during the next sprint?”
Scrum’s strength dwells in its fluidity—it can adapt, change, or react as needed to changing requirements. As the project moves forward, certain requirements and priorities can shift; some increase in significance while others diminish or become totally irrelevant. With such fluidity, the definition of “doneness” becomes vital.
How does the team know when the project is complete? While there are many possible benchmarks, one thing is certain: the definition of “done” must be universal within the team, from the shareholder to the Scrum Master to every programmer, engineer, and tester. Due to variables determined by the scope and nature of each project, it is not possible to devise a catch-all definition, but some examples include:
“The definition of ‘done’ must be universal within the team.”
- All stories are complete
- Functional testing done
- Regression testing done
- Acceptance testing done
- Complete code coverage
- Project metrics are complete
Again, the varied nature of software development makes giving a single definition impossible. This makes it critical for the team to collaborate and agree on what “done” means in their unique environment.
Envisioning Success with Magenic’s Brand of Scrum
If Scrum grants architecture to Agile development, then Magenic goes one step further with our own brand of Architected Scrum and formal process of Envisioning. Envisioning attempts to wrangle the uncertainty of the project’s outcome by working with the shareholders to mitigate risk and determine high-level requirements.
For example, Magenic uses Envisioning to sharpen the definition of “done” by setting the project’s goals and terms of acceptance. The team revisits these goals during the project to assess progress as well as at the end to ensure completion and perform self-evaluation.
Envisioning includes seven activities designed to construct up-front architecture to Scrum methodology:
- Engagement Planning
- Stakeholder Identification
- Master Data Identification
- Major Workflow Identification
- Integration Point Identification
- Service Level Identification
- Quality Needs Identification
Engagement Planning involves the executive stakeholders and sponsors from both Magenic and the customer, during which they define their relationship and collaborate on the project’s implementation. Though this includes functional information, it also serves to establish teamwork and lines of open communication.
As its name implies, Stakeholder Identification identifies the stakeholders to the project team for the sake of constructing user personas that will allow feature tracking against the relevant user communities.
Master Data Identification isolates organizational data sources, the validation rules that will govern them, and how they interact with the application. Envisioning examines the interaction between these sources and the application to forge a framework that supports that interaction with the appropriate data access model and databases.
Major Workflow Identification identifies the paths that end users will take through the application so as to determine and document desired features and outcomes.
- Define the customer relationship.
- Encapsulate understanding of the project:
Source: Magenic University: Magenic
Integration Point Identification exists for all applications that will interact with third-party systems. It defines the number and complexity of these interactions, each of which will affect the application’s requirements and architectural structure.
Service Level Identification limits itself to defining non-functional requirements like availability, scalability, security, and performance to refine the risk profile. All are documented and incorporated into the planning process.
Quality Needs Identification identifies the application’s needs regarding the best methods for quality assurance. Different projects benefit from different QA methodologies; this step defines a set of QA reporting and processes and matches them to the application’s needs.
The Envisioning process produces five artifacts designed to reinforce the philosophy of EDUF with structure and planning:
“Magenic augments Scrum’s architecture with Envisioning.”
- Engagement Plan
- Project Charter
- Quality Plan
- MEWS Estimate
- Magenic Project Site
The Engagement Plan establishes the ground rules and processes to be followed by the team during development. It identifies and documents such things as escalation, change and configuration management, risk management, communication channels, and meeting schedules.
The Project Charter details Magenic’s understanding of the application’s high-level functionality, architectural constraints, technological challenges, and risk. Magenic updates this document as the cycle matures.
The Quality Plan begins at project outset and seeks to choose the best available quality assurance methods to meet the particular demands of the project and its risk profile.
Since Scrum’s iterative development and potentially costly schedule of regular meetings may push some clients out of their comfort zone, Magenic’s format of Architected Scrum calls for more up-front design with its formal process of Envisioning. Envisioning attempts to take the edge out of uncertainty and risk during the initial planning stages with six specific activities. These activities identify pertinent quality assurance methods, stakeholder needs and concerns, and other key areas before embarking on the iterative development process. All of the activities draw on Magenic’s experience developing projects that share qualities with the shareholder’s project.
The activities, in turn, produce five key artifacts that further refine the ground rules for development. These artifacts isolate, analyze, and document elements such as the following:
- Risk factors
- Meeting schedules
- Technological challenges
- Architectural constraints
- Magenic’s understanding of the application’s functionality
- Most efficient means of quality assurance
Source: Magenic University: Magenic
The Magenic Estimation Work Sheet (MEWS) tool uses the functionality and user stories identified during the Envisioning meetings to generate a high-level estimate. This estimate produces additional detail and risk factors to the environment and the project’s scope. These, in turn, are used to build a comprehensive estimate based on Magenic’s experience developing similar applications.
Each of Magenic’s projects receives a Magenic Project Site based on collaboration technologies such as Team Foundation Server, SharePoint, and Lync Server. The goal is to augment the collaborative effort while providing transparency and traceability to all phases of development.
Up-front design attempts to manage risk by proposing a budget, establishing deadlines, and defining all of an application’s requirements before proceeding with development. While this is understandable, it may prove to be counterproductive by failing to deliver a profitable product because of its inflexibility. Big Design Up Front (BDUF) prevents realization of deliverables until the end of the process. Employing this methodology exposes companies to the risk of discovering that they have overlooked vital requirements and wasted time and money on others that prove irrelevant. This worst-case scenario results in an application with lower perceived value to the customer.
Agile development adopts the iterative cycle that allows developers to determine the application’s vital features early and build outward. Requirements are uncovered along the way. Agile tends to individuals, communication, and adaptability over rigid pre-planning and time constraints, but most shareholders demand some level of up-front design before assuming risk.
As an Agile practice, Scrum retains the notions of flexibility and collaboration by finding the middle ground between planning and malleability. The Agile mindset benefits from discipline and organization. While Scrum adopts the tenets of iterative development and individual interaction, it does so with a documented schedule of team meetings under the guidance of the Scrum Master. The Scrum Master removes impediments, holds the team accountable, helps the team self-motivate, and ensures that every team member and stakeholder agrees on the benchmarks for determining project completion, or “doneness.”
Magenic augments Scrum’s architecture with its own Envisioning process. This formal, initial step mitigates risk and uncertainty by devoting the early stages of the development cycle to perform the following tasks:
- Determine high-level requirements in the form of major workflows and epic stories.
- Establish lines of communication, teamwork, and a meeting schedule.
- Develop Magenic’s comprehension of the project’s scope, budget, risk, and technology.
- Prepare a plan for optimal quality assurance methodology.
- Identify and analyze data sources, workflow, integration points, and service level.
- Create a project charter, quality plan, Magenic Estimate Worksheet, and a Magenic project site.
Successful development projects evolve throughout the effort, from beginning to end. Envisioning minimizes risk by engaging in enough up-front design to facilitate iterative development and collaboration, yet it also avoids the trap of making a host of dodgy and ultimately costly predictions at the project’s inception. A business’s survival depends on adapting to its customer’s eternally evolving needs; architected Scrum and Envisioning not only expects these changes, but embraces them.