Introduction
The Workforce Investment Act (WIA), authorized in 1998 as the nation’s public workforce development system, has a stated purpose “to consolidate, coordinate, and improve employment, training, literacy and vocational rehabilitation programs in the United States.” Given this mission, how can we determine the WIA programs’ effectiveness over extended periods of time?
That’s the question that was brought to EMSI by several executive directors of local workforce investment boards (WIBs). The directors were looking for a rigorous economic evaluation of their programs—an evaluation that was affordable, repeatable, unbiased, and able to withstand scrutiny from different stakeholder groups. Given our 10-plus years of experience in performing benefit-cost analyses for government-funded programs, EMSI was uniquely qualified to meet these requirements. We accepted the challenge, and we’ve spent the past year developing an economic model designed to assess WIA programs.
The result? The WIA Scorecard.
EMSI’s Scorecard gives workforce leaders the information they need to cost-effectively measure and communicate the effectiveness of their WIA efforts. By leveraging readily accessible data, you can produce a report in just a few weeks, all at a fraction of the cost of unique comparison group studies. We invite you to take a look at the Scorecard (PDF) we produced for South Central Michigan Works! earlier this year. (We also conducted a study for the WIB of Southwest Missouri, which corresponds to the cover image above).
In this article, we want to take a closer look at the Scorecard model and how it works. We’ll cover the particular advantages of benefit-cost analysis, the methodology we use, and the details of interpreting the study results.
Benefit-Cost Analysis
What does the Scorecard’s benefit-cost analysis do? Simply put, it weighs the overall benefits of a program against its overall costs.
The most commonly used measurement is the benefit-cost ratio: total benefits divided by total costs. The simplicity of the benefit-cost ratio makes it both easy to understand and easy to use as a baseline. It’s also a clear and concise metric to communicate to decision-makers. A ratio less than 1.0 means that the costs outweigh the benefits, while a ratio greater than 1.0 means that the benefits outweigh the costs and are generating a payback to the original investors. This simple and effective performance indicator resonates well with board members, congressional representatives, and other stakeholder groups.
Now, you might be wondering why we chose to do a benefit-cost analysis instead of a return-on-investment (ROI) analysis of workforce training programs.
This is because an ROI analysis seeks to measure the returns that accrue to the original investors, but when the government invests in a social good, the societal benefits flow to far more people than just the original investors. As such, performing an ROI analysis becomes impractical. We chose a benefit-cost analysis because it allowed us to capture and quantify all measurable benefits to society, not just those that went to taxpayers.
We chose a benefit-cost analysis because it allowed us to capture and quantify all measurable benefits to society, not just those that went to taxpayers.
You can perform benefit-cost analysis annually to get a year-over-year benchmark of WIB performance. Workforce investment is dynamic; WIBs experience peaks and valleys that reflect regional and national economic conditions. And when workforce and industry need change, WIBs implement new strategies to continue bringing workers and industry together. Ongoing benefit-cost analysis can give a realistic picture of the effectiveness of WIA funding in connecting jobseekers with employers.
Methodology
How does EMSI measure benefits and costs of WIA programs, and where do we get the data? WIA funds are divided into three categories: Adult, Dislocated Worker, and Youth. Much of the data comes from the Workforce Investment Act Standardized Record Data (WIASRD) file, which the US Department of Labor makes available to the public. The rest of the data comes directly from the WIB or from the state’s unemployment insurance (UI) wage records.
The total benefits calculation process has four steps:
- Step 1: We calculate the average earnings change between the pre-program earnings and post-program earnings of participants who find and retain employment during the analysis year. We then account for inflation, project the marginal earnings benefit 10 years into the future, and discount to account for the time value of money.
- Step 2: We adjust the future marginal earnings stream to account for the counterfactuals. These counterfactual adjustments include the probability that participants would have been able to find employment without WIA treatment, the length of time before the effect of WIA treatment wears off, and the rate at which workers leave the workforce over time.
- Step 3: We estimate the direct non-labor income effects that occur as participants generate additional income in the economy through their employers’ increased capital investment.
- Step 4: The first three steps give us total benefits. Now we divide these benefits by the total costs to generate a benefit-cost ratio for each WIA program (Adult, Dislocated Worker, Youth).
EMSI designed the benefit-cost analysis to be conservative. The Scorecard also includes a sensitivity analysis to test the sensitivity of the results to variations in the input assumptions.
Interpreting the Results
Now, we’ve seen that if a WIB’s overall score is above 1.0, the programs’ benefits outweigh the costs, which indicates a feasible investment. But what if a WIB’s overall score is below 1.0? Does this signify a bad investment.
Here we’d like to point out that the expectation for government-funded programs to recover all costs might be misguided. Worthwhile public projects frequently generate benefit-cost ratios which, compared to those in the private sector, seem rather measly. (Benefit-cost ratios in the public sector that dip as low as 0.3 are normal, even expected.) This is because the government’s role is to provide services that the public wants—services which pay back in ways other than money but which the business sector may find unprofitable. Public parks, for example, require hefty funds and don’t generate nearly enough monies to cover the cost of supporting them, but they yield lots of non-quantifiable benefits which visitors enjoy. Similarly, WIA produces many benefits that don’t necessarily translate to jobs and income, but that still have a positive impact on society.
Even so, there are ways to increase the overall benefits. WIBs can improve their metrics by increasing the number of people who find employment, helping people find higher-paying jobs, and ensuring that people retain their jobs for longer periods of time. Efforts like this will bear out in the benefit-cost analysis of EMSI’s Scorecard.
WIBs can improve placement rates, retention rates, and average earnings even more by upping their interaction with the industries and businesses that hire WIB participants. Local businesses can communicate their skill requirements to WIBs, and WIBs can tailor their jobseeker and training efforts to meet these workforce needs. This will allow WIBs to both proactively train and place workers into well-paying jobs and stimulate economic growth through the higher productivity of local businesses that are now better able to fill their job orders.
Many WIBs use EMSI’s labor market tools to determine which industries they should target for jobseeker placement. EMSI’s Analyst, for example, provides the latest region-specific industry and occupation data, along with workforce skills and competencies. WIBs use this data to spot trends in workforce needs and to actively engage and form relationships with local businesses.
Conclusion
EMSI designed the WIA Scorecard analysis for WIBs that desire transparent, repeatable, third-party evaluations of their programs with quick, easy-to-communicate results. Once a WIB has established a baseline with the Scorecard, we can assist their attempts to target specific industries and business. We encourage WIBs to give EMSI the opportunity to help their efforts to improve workforce training programs and better serve both jobseekers and employers.
For more on the Scorecard, contact Jeff Perley at jperley@economicmodeling.com or by calling 866.999.3674. More information can be found at www.economicmodeling.com/wiascorecard.