This publication is broken up into three sections:
TL;DR - For those wanting a quick take
Summary - For those wanting a bit more context and high level points
Article - Main body of work containing fully detailed article and explanations that you might want to consume over several readings
TL;DR
Combining Reinertsen's "Principles of Product Development Flow" and Kersten's "Project to Product" with the Logic Model offers a comprehensive approach to evaluate and enhance productivity in product development roles.
You can apply this integrated framework to roles in product development, considering their specific contributions by aligning roles like Product Managers, Designers, and Product Developers with business goals to drive value.
This integrated framework emphasises economic decision-making, focusing on value creation, cost-effectiveness, and resource allocation.
Measure flow metrics (velocity, time, load, efficiency) to optimize the value delivery process.
This integrated framework fosters cross-functional collaboration, continuous learning, and adaptability.
Finally and most importantly to prevent metric manipulation you need meaningful, and diverse metrics with a long-term outcome focus. This requires ethical data practices, organisational transparency, and a culture of innovation.
Summary
The integration of Donald G. Reinertsen's "Principles of Product Development Flow" and Mik Kersten's "Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework" offers from my perspective a comprehensive approach to evaluating and enhancing product developer productivity across various roles, including Product Managers, Designers, and Product Developers.
This integration emphasizes several key aspects:
Alignment with Business Goals: The framework encourages aligning software and hardware development efforts with overarching business objectives. It evaluates how effectively Product Managers, Product Designers, and Product Engineers contribute to delivering products that meet customer needs and drive business success.
Leadership and Accountability: Both frameworks stress the importance of leadership and accountability within each role. Leaders set an example by focusing on value, optimizing flow, and promoting a culture of continuous improvement.
Business Value and Metrics Elements: Economic aspects of product development are highlighted, evaluating the productivity of these roles based on their contribution to value creation. Decision-making that leads to high market value products is a crucial factor.
Value Stream Mapping and Flow Efficiency: Reinertsen's principles emphasize optimizing the flow of work in product development. Applying value stream mapping to visualize the process and measuring flow efficiency helps assess the efficiency of these roles.
The Four Key Flow Metrics: Kersten's four key flow metrics – flow velocity, flow time, flow load, and flow efficiency – are used to assess the productivity of Product Managers, Designers, and Product Engineers. These metrics measure how quickly requirements are prioritized, designs are created, and features or fixes are delivered.
Preventing the manipulation of metrics within this framework is essential to ensure accurate assessments. Several measures can be taken, including:
Focus on Meaningful Metrics: Choosing metrics that truly reflect the quality and value of work rather than easily manipulated ones.
Diverse Metric Set: Using a combination of metrics that provide a comprehensive view of the development process, making it harder to manipulate a single metric.
Transparency and Education: Ensuring developers understand the metrics, how they're measured, and why they matter, discouraging manipulation.
Team Collaboration: Fostering a culture of collaboration and shared goals within development teams to reduce the incentive for metric manipulation.
Regular Review and Adaptation: Continuously reviewing and adapting metrics to stay ahead of potential gaming strategies.
Focus on Long-Term Outcomes: Emphasizing long-term outcomes and quality over short-term gains.
Peer and Management Reviews: Introducing mechanisms for peer reviews and management oversight.
Balanced Scorecards: Implementing balanced scorecards that consider multiple dimensions of performance.
Automated Data Collection: Using automated tools for data collection to reduce manual manipulation.
Ethical Leadership: Cultivating an ethical leadership culture that encourages honesty and integrity.
Feedback Mechanisms: Establishing feedback loops where developers can discuss concerns related to metrics.
Additionally, the framework I propose integrates the Logic Model to create a more comprehensive and structured approach to evaluating product development. It aligns inputs (resources and requirements), activities (value delivery and process steps), outputs (deliverables, products and services), outcomes (flow metrics and impact), and goals/objectives (strategic alignment) within the Logic Model.
This alignment ensures that the product development process is purposeful, efficient, and most importantly strategically aligned with organizational objectives.
Role-specific ideas for evaluating productivity in various product development roles are also included:
Product Managers are evaluated based on their ability to define clear requirements, manage priorities, and align with customer needs and business objectives.
Designers are assessed on their contributions to user experience and product quality.
Software Developers are evaluated on delivering features and fixes efficiently and managing flow metrics.
Data Analysts are assessed on their ability to extract insights from data and provide actionable information.
Data Scientists are evaluated on their contributions to advanced data analysis and predictive modeling.
These evaluations are aligned with the broader framework, emphasizing economic value, data-driven decision-making, and continuous improvement. Ethical considerations and transparency are also emphasized in data collection and decision-making processes.
By integrating these principles, the integrated framework I propose offers a comprehensive approach to measuring and enhancing productivity across different product development roles while promoting ethical behavior and data-informed practices.
Article
First pass of a sound product development framework
Combining the principles and frameworks from Donald G. Reinertsen's "Principles of Product Development Flow" and Mik Kersten's "Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework" can provide a comprehensive approach to evaluating and enhancing product developer productivity, encompassing Product Managers, Designers, and Product Developers.
Here's how you can integrate their concepts:
Alignment with Business Goals:
Apply Kersten's idea of aligning software and hardware delivery efforts with business goals to all roles. Evaluate how well Product Managers, Designers, and Product Engineers contribute to delivering products that meet customer needs and drive business success.
Leadership and Accountability:
Both frameworks emphasise leadership and accountability within each role. Leaders can set an example by focusing on value, optimizing flow, and encouraging a culture of continuous improvement.
Business Value and Metrics Elements:
Focus on Economics and Value: Reinertsen's principles highlight the economic aspects of product development. Evaluate the productivity of Product Managers, Designers, and Product Engineers based on their contribution to value creation. Are they making decisions that lead to products with high market value?
Value Stream Mapping and Flow Efficiency: Reinertsen's principles emphasize the importance of understanding and optimizing the flow of work in product development. Applying the concept of value stream mapping to visualize the end-to-end process, identifying bottlenecks and areas of waste is a great starting point. Do you know how work flows through your own product development process and what are the key touchpoints with the product development team? Kersten's Flow Framework introduces the concept of flow efficiency. Use this to measure how efficiently work moves through the development process. Apply this to all roles - Product Managers, Designers, and Product Engineers - to ensure smooth collaboration and minimized delays.
The Four Key Flow Metrics: Both frameworks can be combined by using the four key flow metrics introduced by Kersten: flow velocity, flow time, flow load, and flow efficiency. Apply these metrics to assess the productivity of Product Managers, Designers, and Product Engineers.
These initial measurement ideas are less than perfect but could involve you measuring how quickly Product Managers can prioritize and provide clear requirements, how efficiently Designers create designs, and how Product Engineers deliver features or fixes linked to business vakue.
A key point to make here is that a short flow time is not indicative of high quality and/or high output does not necessarily mean product development teams are delivering value. See article here on how product development teams need to think about information and value creation. These metrics should be part and parcel of a much larger balanced set of metrics that need to be considered when evaluating performance.
Individual and Team Elements:
Cross-Functional Collaboration and Value Streams: Reinertsen emphasises the need for cross-functional collaboration. Recognize that Product Managers, Designers, and Product Developers form critical parts of a value stream network. Optimize the interactions and handoffs between these roles to streamline the flow of work. All role players are needed for high-performing product development teams.
Continuous Learning and Improvement: Both frameworks stress the importance of continuous learning and improvement. Encourage Product Managers, Designers, and Product Engineers to learn from experiements, and incorporate feedback. This culture of improvement can boost productivity.
Transformation and Adaptation: From Kersten's framework, adaptability is crucial. Recognize that Product Managers, Designers, and Product Engineers must stay open to change and be willing to embrace new tools and methodologies to remain productive in a dynamic environment.
Transparency and Metrics Education: Emphasize transparency regarding metrics and their significance. Educate Product Managers, Designers, and Product Engineers on how the metrics are measured, why they matter, and how they collectively contribute to the success of the organization.
Regular Review and Adaptation: Continuously review and adapt the chosen metrics and practices for evaluating each role's productivity. Regularly revisit how the principles from both authors are applied to ensure they remain effective.
By integrating the principles from both Donald G. Reinertsen and Mik Kersten, you can create a holistic approach to evaluating and enhancing the productivity of Product Managers, Designers, and Product Engineers aka Product Developers.
Ways to prevent manipulation of metrics
Preventing product developers from "gaming" the metrics within a framework is crucial to ensure that the metrics accurately reflect productivity and do not lead to counterproductive behaviors. Here are some measures that can be taken to mitigate the risk of product developers gaming the metrics:
Focus on Meaningful Metrics: Choose metrics that truly reflect the quality and value of the work being done. Metrics that are easily manipulated or don't align with actual outcomes are more likely to be gamed. Prioritize metrics that measure real progress and contribution. This will require debate and discussion to get to a point where alignment is clear on how teams or functions contribute to the greater whole of an organization. Bad metrics like the number of lines of code produced or bugs fixed per day metrics in isolation are not good measures and can easily be gamed.
Diverse Metric Set: Instead of relying on a single metric, use a combination of metrics that provide a comprehensive view of the development process. This makes it harder for developers to manipulate one specific metric to their advantage.
Transparency and Education: Make sure that developers understand the purpose and implications of the metrics being used. Transparently communicate how metrics are measured, what they represent, and why they matter. Educating developers about the potential pitfalls of gaming metrics can discourage such behaviors. Ideally co-create metrics with the product development team that are meaningful and valuable to the team. A team that looks after a payments system would ideally have metrics associated to the cost of downtime and the value generated or enabled through their system services.
Team Collaboration: Foster a culture of collaboration and shared goals within development teams. When team members work together toward a common objective, there's less incentive to manipulate metrics for personal gain.
Regular Review and Adaptation: Continuously review and adapt the chosen metrics to ensure they remain relevant and effective. As developers become familiar with the metrics, they might find ways to exploit them. Regularly reassess metrics to stay ahead of potential gaming strategies.
Focus on Long-Term Outcomes: Emphasize the importance of long-term outcomes and quality over short-term gains. Developers should be encouraged to focus on delivering value to customers and the organization rather than solely optimizing for individual metrics.
Peer and Management Reviews: Introduce mechanisms for peer reviews and management oversight. Regular reviews by peers and managers can help detect unusual patterns or behaviors that might indicate gaming of metrics.
Balanced Scorecards: Implement balanced scorecards that consider multiple dimensions of performance, including qualitative aspects such as collaboration, innovation, and problem-solving. This prevents developers from hyper-focusing on a single metric.
Automated Data Collection: Use automated tools to collect data for metrics whenever possible. This reduces the opportunity for developers to manually manipulate data to alter metrics.
Ethical Leadership: Foster an ethical leadership culture that encourages honesty, integrity, and ethical behavior. When leaders exemplify ethical conduct, it sets the tone for the entire team.
Feedback Mechanisms: Establish feedback loops where developers can openly discuss concerns or challenges related to the metrics. This provides an avenue for addressing potential issues before they escalate.
Encourage Improvement Over Perfection: Encourage developers to focus on continuous improvement rather than trying to achieve perfect metrics. When improvement is the goal, developers are less likely to engage in short-term manipulation.
By combining these measures, organizations can create an environment that discourages gaming of metrics and promotes genuine efforts to deliver value and improve the development process. It's important to recognize that no approach is foolproof, but a combination of transparency, education, collaboration, and ethical leadership can significantly mitigate the risk of gaming.
Second pass of a product development framework incorporating the Logic Model
Inputs (Resources and Requirements):
In the Logic Model, inputs represent the resources required to implement a program.
In the context of Mik Kersten's Flow Framework, these inputs include:
Funding: Budget allocated for product development activities.
Human Resources: Skilled Product Managers, Designers, Software Developers and Engineers.
Tools and Technology: Software tools, platforms, and technologies used for development.
Requirements: Clear and well-defined product requirements provided by stakeholders.
Incorporating Reinertsen's principles of economic decision-making into the assessment of inputs into the Logic Model leads to:
Considering not only the availability of resources but also their economic efficiency.
Evaluating the allocation of funding, ensuring that it maximizes the economic value of product development activities.
Assessing the cost-effectiveness of human resources, tools, and technology investments, optimizing resource utilization while aligning with economic objectives.
Activities (Value Delivery and Process Steps):
Activities in the Logic Model are the specific steps taken to achieve program objectives.
In the Flow Framework, activities translate to the value delivery process, which aims to optimize the flow of work. These activities include in a product development context:
Product Management: Defining and prioritizing product features based on customer needs.
Design and UX: Creating user-centered designs for optimal user experience.
Software and/or hardware Development: Writing code and developing products.
Testing and Quality Assurance: Ensuring the software and/or hardware meets quality standards.
Applying Reinertsen's economic principles to the evaluation of activities in the logic model lead to:
Emphasising the economic value generated by each activity within the value delivery process.
Assessing whether activities are aligned with customer value and contribute to economic decision-making.
Identifying opportunities to reduce waste, optimize resource allocation, and enhance economic efficiency throughout the development process.
Outputs (Deliverables and Products):
Outputs in the Logic Model are the tangible results of activities.
In the Flow Framework, outputs align with the products and deliverables resulting from the value delivery process, such as:
Product Features: Tangible enhancements to the software product.
User Interface Designs: Visual representations of the user experience.
Codebase Updates: Revised and new code segments in the software.
Hardware Prototypes: Actual working production grade hardware samples.
Integrating economic considerations into the evaluation of outputs from Reinertsen's economic principles to the evaluation of outputs in the Logic Model leads to:
Measuring the economic impact of product features, user interface designs, and codebase updates.
Assessing whether these outputs contribute directly to economic objectives, such as revenue growth or cost reduction or risk mitigation.
Identifying areas where outputs can be refined to maximize economic value and minimize unnecessary costs.
Outcomes (Flow Metrics and Impact):
Outcomes in the Logic Model represent the changes or benefits resulting from outputs.
In the Flow Framework, outcomes can be aligned with the flow metrics and the impact on product development efficiency and quality:
Flow Velocity: Increased speed of delivering features and fixes.
Flow Time: Reduced time taken for features to move through the development process.
Flow Efficiency: Improved efficiency by reducing bottlenecks and waste.
Business Impact: The overall impact on the organization's objectives, such as increased customer satisfaction, revenue growth, or market share.
Aligning Reinertsen's focus on economic outcomes with the evaluation of flow metrics and impact leads to.
Evaluating the economic implications of flow velocity, flow time, flow efficiency, and overall business impact.
Measuring how improvements in flow metrics translate into economic benefits, such as reduced time-to-market or increased customer satisfaction.
Using economic criteria to prioritize outcomes that have the most significant impact on organizational goals.
Goals/Objectives (Strategic Alignment):
Goals in the Logic Model represent the broader objectives of a program.
In the Flow Framework, goals align with the strategic alignment of product development efforts with business objectives:
Business Objectives: The overarching goals of the organization, such as entering new markets, increasing customer retention, or achieving digital transformation.
The Logic Model when aligned to Reinertsen's principles of economic decision-making for broader business objectives leads to:
Emphasising the economic rationale behind strategic alignment, emphasizing the economic value created by achieving these goals.
Using economic metrics to track progress toward strategic objectives, allowing for data-driven decision-making in support of economic goals.
By aligning Mik Kersten's Flow Framework and Reinertsen's economic framework with the Logic Model, organizations can systematically plan, implement, and gain a more comprehensive approach to evaluating product development initiatives.
This alignment ensures that the product development process is purposeful, efficient, and strategically aligned with the organization's objectives, while also allowing for clear measurement and assessment of outcomes and impact.
This integrated framework emphasizes the economic aspects of software development, guiding organizations to make decisions that maximize value, minimize waste, and align with strategic economic objectives.
It allows for a holistic assessment of productivity while considering the economic implications of every component within the development process, ultimately leading to more economically efficient and impactful product development efforts.
This holistic framework covers the full value chain and can be used to create balanced product development team scorecards. Below are some ideas on how to measure productivity within different product development team roles.
Role Specific Ideas to evaluate productivity
Product Managers:
Evaluate the productivity of Product Managers by considering their role in defining clear requirements, managing priorities, and aligning with customer needs and business objectives.
Measure their ability to create effective value streams and make data-driven decisions.
Integrate the Logic Model's outcomes with Reinertsen's emphasis on economic decision-making for Product Managers.
Product Managers should ensure that their decisions maximize value while minimizing waste and unnecessary features.
Designers:
Assess Designers' productivity by examining their contributions to user experience and overall product quality.
Measure their ability to create designs that not only enhance flow efficiency but also resonate with end-users e.g., Google HEART/GST measurement framework.
Incorporate data analysis to gather insights on user behavior and preferences. Data-driven design decisions can lead to more impactful and user-centered designs, aligning with both the Logic Model's outcomes and Kersten's Flow Framework.
Software Developers:
Evaluate Software Developers' productivity based on their role in delivering features and fixes efficiently.
Measure their management of flow velocity, flow time, and flow efficiency. Incorporate data analysis to monitor code quality, identify bugs, and optimize performance e.g., DORA and SPACE metrics.
Data-driven insights can help developers address issues proactively, leading to improved product quality and customer satisfaction. The integration of data analysis aligns with both the Logic Model’s outcomes and the Flow Framework’s focus on flow efficiency.
Data Analysts:
Integrate Data Analysts into the evaluation process by recognizing their role in extracting meaningful insights from data.
Evaluate their ability to analyze and visualize data to provide actionable information for informed decision-making.
Align their contributions with the Logic Model's outcomes, emphasizing their impact on data-driven strategies and improved decision outcomes.
Data Scientists:
Include Data Scientists in the evaluation framework, focusing on their contributions to advanced data analysis, predictive modeling, and algorithm development.
Evaluate their role in deriving insights that drive innovation, optimize processes, and create value.
Align their efforts with both the Logic Model's outcomes and the Flow Framework's emphasis on continuous improvement through data-driven insights.
Alignment with Business Objectives:
Within the Logic Model, ensure that the goals/objectives align with the broader business objectives for all roles, including Data Analysts and Data Scientists.
Their contributions should be aligned with achieving organizational goals, enhancing the Logic Model's strategic alignment aspect.
Continuous Improvement and Learning:
For all roles, emphasize the importance of continuous improvement, learning, and adaptation.
Apply the concepts from both Reinertsen and Kersten to foster a culture of innovation, data-driven decision-making, and ongoing enhancement of skills and practices.
Ethical Considerations and Transparency:
Ensure that data collection, analysis, and decision-making adhere to ethical guidelines and transparency standards, safeguarding the integrity of the evaluation process.
By expanding the role evaluation section to include Data Analysts and Data Scientists, the combined framework becomes even more comprehensive, addressing a wider range of roles within the context of product development.
This integrated approach enables organizations to assess and enhance productivity across Product Managers, Designers, Software Developers, Data Analysts, and Data Scientists, all while considering their individual contributions within the broader framework of value-driven and data-informed product development.
Postscript
Before you go, please could you do the following?
Subscribe
Share
Survey
Star
If you got value from reading the article a star liking would be highly appreciated!
Some Resources
Gergely Orosz (Part. 1 of 2) structured critique of McKinsey & Company’s proposals
Kent Beck (Part 2 of 2) structured critique of McKinsey & Company’s proposals