Broaden Data Sources for Program Review

It's imperative for workforce education programs to calibrate for the future. One of the best tools for calibration is the venerable Program Review. Here are a few ideas to ensure your Program Review is adequately and accurately responding to the anticipated future.

Broaden Data Sources

Traditional review processes tend to focus on institutional data, labor market data, faculty insight, student experience, and advisory committee perceptions. These are important measures; however, they largely measure what's "inside the box." To look beyond the program lens, the Review can seek out several additional data sources. Described below, the quantitative sources (databases, etc) are readily available while the qualitative sources (survey and interviews) require additional effort.

1. Understand potential enrollees. Workforce education programs are selling an educational product (i.e., skills and knowledge codified by a degree or certificate), and it's important to understand as much as possible about the consumers of this product. Program Reviews could access reliable, contextual data that answers questions like: Who are potential enrollees? What motivates them? What do they want? What do they need? How do they want to access your product? How do they communicate? Sources of data required to answer these questions include K12 demographics, referring partner demographics, US Census, direct inquiry with high schools and referring partners, and direct engagement with potential enrollees. Student enrollment is the lifeblood of workforce education, and the Program Review is well-positioned to innovate change to ensure sustainable enrollment.
2. Align skills "taught" with those "sought." Workforce education is structured to respond to the needs of local and regional job markets. To maximize success with this mission, the Program Review could collect and analyze the following data points: 1) What jobs students are being trained for? 2) What jobs are graduates actually getting? 3) Who are the employers of these jobs? 4) What skills and experiences are listed in current job postings? 5) How do employers describe the competencies they seek in a successful job applicant? 6) How do program learning outcomes overlap/gap with job postings & described competencies? Sources for this data include job postings, direct engagement with employers, program & course outcomes, and graduate profiles.
3. Anticipate trends of emerging and evolving occupations. Curricular change is a constant task for workforce education programs. As technologies, techniques, and practices evolve in the workplace, workforce education programs must evolve, too. Anticipating future trends requires an understanding of current ones, and there are rich sources of labor market data that answer questions like: 1) What other occupations value this program's learning outcomes? 2) What skills are in-demand? 3) How are required skills changing? 4) How have aligned occupations changed? Based on these trends, the Program Review can frame anticipated changes by combining insights collected from employers and industry leaders. Building and maintaining a data-driven vision can lead to better decisions and effective change.

While any one of these ideas (if done well) would help a Program Review look forward, the combination of all three could provide a clear and doable approach to be a responsive program supported by sustainable enrollment and close alignment with the workforce. In many colleges, these forward-looking efforts are addressed by a central resource like a department or office, which can be far from the program. By including these forward-looking activities in the Program Review, they are more likely to produce responsive, acceptable, and ultimately successful change. Clearly an inherent barrier to these ideas is the increased level of data to be handled and processed into a usable resource. This barrier can be overcome by developing/tapping internal capacities or hiring external help.