Simple Analysis Method for Assessment of PEOs Using Limited Survey Data

Alumni and employer surveys are among the assessment tools often used to determine the extent to which a program's graduates are able to successfully pursue the types of careers envisioned in the program educational objectives (PEOs). A major problem with surveys is the very low response rate. However, while the number of responses to the individual surveys in a given year results in limited statistical significance, it is a case whereby the data as a whole, especially interpreted and reinforced by knowledge gained from a variety of other sources, paints a reasonably consistent and coherent picture. In this paper, a simple analysis method is described that is based on tools often used in market research. Specifically, every question on the alumni survey is considered as an opportunity to measure success or failure of the alumni's perceived obtainment of the PEO. This type of analysis is called "top-box" and "bottom-box" analysis and is often used for reporting customer satisfaction survey data. Finally, the data is accumulated in a manner often done in trying to predict reliability for complex systems when only a limited number of systems are built. Specifically, we utilize a multi-year moving window and aggregate the data.