Measuring What Matters: The Importance of Measuring Effectiveness Over Performance

by Allison Barlow & Julian Beverly

Today’s organizations invest significant time and resources defining and implementing strategic priorities and goals. Leaders set the vision, managers drive its execution, and frontline workers bring it to life. Yet organizations rarely apply the same rigor when determining whether those efforts actually work. This gap is especially evident in organizational change management, where speed equals performance while the impact of speed is rarely measured or quantified.  

Organizational change management is the practice of guiding and implementing change to achieve new and innovative priorities, goals, or strategies for the betterment and advancement of the organization. Internally, organizational change management teams brainstorm and document changes to help increase productivity, profitability, and/or customer satisfaction. While a lot of thought goes into planning for the organizational transformation, organizations often come up short when measuring the effectiveness of these changes.  

Organizational transformation and change efforts are measured by two interconnected and complementary concepts: measures of performance (MOPs) and measures of effectiveness (MOEs). MOPs focus on helping teams track performance, like activities and outputs (i.e., did we accomplish the task?). MOEs focus on effectiveness, like outcomes and results (i.e., did the task yield the expected result?). Because high performance can coexist with poor effectiveness, it is critical to delineate between the two; for example, MOPs ensure that teams provide employees with the tools and knowledge required to support a change. On the other hand, MOEs confirm that the tools are effective and that workforce has embraced the tools to achieve a change. 

Effectiveness measures provide the foundation for data-driven decision-making during organizational transformation and change efforts. Without getting a clear picture of whether employees are adopting a change to achieve a priority or goal, leaders remain unaware of the reality on the ground and whether the organization has successfully implemented a change. Project dashboards tend to overemphasize MOPs and operational measures because technology helps track clicks and project milestones. To measure true effectiveness, organizations must collect both quantitative and qualitative data to measure the impact of a change. 

To measure change effectiveness, organizations should follow these steps: 

To start, the organization must identify the most impactful MOPs and MOEs for determining whether the change effort is progressing toward the intended outcomes. These measures should include initial MOPs (e.g., rolling out a pilot, publishing a policy, launching a communications campaign, etc.) as well as MOEs (e.g., reduction of time to complete a function as a result of the piloted method, reduction of incidents as a result of the policy, improved customer or workforce sentiment as a result of the communications campaign, etc.). 

Next, the organization needs to develop a data collection plan to ensure the data identified to inform the MOPs and MOEs is collectible in a reasonable and consistent manner.  

The organization must then establish a baseline. A baseline defines the current state of the organization before a change initiative begins. Once the organization has established a baseline foundation, it can leverage the baseline to accurately measure the effectiveness of a change.  

Once the organization establishes a baseline, it must create an adaptable framework to measure the change and track MOPs and MOEs, with a strategic focus on MOEs. Developing an adaptable framework, like a dashboard or data visualization tool, supports leaders with data-driven decision-making and enables the organization to adjust as it tracks change progress.  

Following the establishment of a measurement framework, the organization must communicate the importance of the change to stakeholders so they understand why the change is happening and what outcomes are expected.  

When organizations fail to define effectiveness measures, they default to measuring what is easiest to quantify: performance. To quantify true transformation and change management, leaders must shift their focus from simply measuring performance to measuring effectiveness. Measures of effectiveness equip organizations with the necessary and appropriate aperture to evaluate whether the goals, priorities, and initiatives that were set forth brought tangible value and yielded a measurable outcome. Prioritizing effectiveness over performance ensures that change initiatives are not just efficiently executed, but meaningfully and successfully adopted. As organizations continue to optimize their operations by developing strategic priorities, achieving true organizational change management will require shifting the focus to measure what matters—from simply tracking performance metrics to comprehensively measuring the actual effectiveness of those changes.