Today, one critical professional skill is being able to show the accountability of various processes and functions. Senior executives and other stakeholders can question the value of a variety of programs, projects, and processes, but professionals with ROI Methodology knowledge can show how value can be measured and reported in credible ways.
The ROI Methodology can be described by considering five major elements:
|1. Reaction & Planned Action||Measures participant satisfaction with the program and captures planned actions|
|2. Learning||Measures changes in knowledge, skills, and attitudes|
|3. Application and Implementation||Measures changes in on-the-job behavior and progress with application|
|4. Business Impact||Captures changes in business impact measures|
|5. Return on Investment||Compares program monetary benefits to the program costs|
The evaluation levels categorize data, reporting a chain of impact as reaction leads to learning, to application, to impact and to return on investment.
The Phillips ROI Methodology™ model provides a step-by-step process for collecting data, summarizing and processing data, isolating the effects of programs, converting data to monetary value, and capturing the actual ROI.
The Phillips ROI Methodology™ is applied through a 10-step process.
First, the planning is initiated and the specific business drivers of the solution are identified. Discussion and decisions revolve around how the solution satisfies the business drivers. Business measures are clearly identified. The objectives are established/revised to ensure that stakeholders agree on the application/behavior change and the business impact measures to be influenced.
The purpose of the evaluation should be clearly defined and baseline data should be developed/collected. If the purpose is to calculate the ROI, the entire ROI Process should be followed. If the purpose is only to determine behavior change, then the evaluation can stop short of collecting business impact data and calculating the ROI. If the purpose is to determine business impact, then data will be collected at all levels. This step also includes determining the data collection strategy and developing the detail planning documents that specify how steps 3 through 10 can be carried out.
Step 3 begins the implementation of the data collection strategy that was planned in the previous step. In step 3, the client organization usually collects the L-1 and L-2 data during the solution implementation (satisfaction/planned action and learning). This data is later reported along with follow-up data from step 4.
In step 4, application/behavior change and business impact are collected. Business impact data is converted to monetary values to calculate the ROI. Throughout the process, data is collected at all levels to show a chain of impact up to the highest level that satisfies the purpose of the study.
In step 5, the data analysis phase of the process begins. The effects of the solution are isolated to determine the extent that the business measures were influenced by the solution.
This step is applied when the purpose of the evaluation includes calculating the ROI. If stakeholders have determined that there is no interest in the ROI calculation for a specific initiative, then the business impact and behavior change data is reported minus the calculation.
Step 7: Identify Intangible Benefits
Step 8: Capture Costs of Solution
Step 9: Calculate the Return on Investment
Step 10: Develop, Repost and Communicate Results
Data from step 7 is reported along with business metric improvements, as well as barriers and enablers to implementation/behavior change. Any improvement in behavior and business metrics influenced by the solution (isolation) is reported in step 10. When the ROI is calculated (step 9), the costs (step 8) are compared to the benefits that are converted to a monetary value from step 6. Additionally, all of the data from steps 3, 4, 5, and 7 are also reported.
Conclusions and recommendations are also reported. Conclusions address information such as what caused the results, and what worked and what did not work. Recommendations address next steps and how the findings can be used to implement improvement.
Throughout the process of a follow-up study, data on behavior is collected. Behavior change is one of the key variables that helps determine if or how much the business metrics improve. Why behavior does or does not change and how it changes is also important. Data collection instruments and methods are carefully planned and developed to collect the most credible data from the most reliable sources to determine the contribution.