Engineering an Education
Pitt professors measure effectiveness of engineering curriculum

Mary Besterfield-Sacre, assistant professor of industrial engineering in Pitt’s School of Engineering, was one of several engineering professors studying curriculum innovations at dozens of engineering schools around the country. The group recently completed an NSF-sponsored study, which advocates multi-source assessment and feedback as a primary measurement process.

Engineering education is changing, as any recent visitor to Pitt’s School of Engineering can attest. With recent construction of the Frank Mosier Learning Center and the freshman engineering classroom, Pitt’s engineering program is undergoing a facelift — literally and figuratively — incorporating physical and curricular changes that emphasize more teamwork, active learning, multidisciplinary coordination, and the application of knowledge to real-world problems.

As in many engineering schools across the country, Pitt’s engineering faculty have responded to calls from the National Science Foundation (NSF), the National Research Council, and the American Society for Engineering Education (ASEE), all of whom issued reports calling for changes in the ways we teach engineering in the U.S.

In each of the past two years, Pitt’s new engineering programs — the curricular overhaul of the chemical engineering department and the integrated freshman program — have received awards from the Carnegie Science Center for their educational innovations. The chemical engineering department’s changes were spotlighted in a feature article in ASEE’s magazine, Prism.

That’s why Larry Shuman, professor and associate dean for academic affairs; Harvey Wolfe, the William Kepler Whiteford Professor of Industrial Engineering; and Mary Besterfield-Sacre, assistant professor of industrial engineering, are studying the innovations at dozens of engineering schools across the country — to develop ways to measure the effectiveness of the changes. They recently completed an NSF-sponsored study which advocates multi-source assessment and feedback as a primary measurement process.

The most difficult part of the project, Besterfield-Sacre said, is that there are no models for them to adopt on the university level.

“On the K through 12 grade levels, they’ve been pursuing outcome-based education for years, but still there’s not a lot of material on measuring these outcomes,” she said.

ABET set 11 outcomes criteria for engineering students, with the mandate that schools not only implement programs that lead to those outcomes, but also that they establish methods of measuring the outcomes. And if they’re not being met, they must quickly adapt programs in order to meet the outcomes.

Part of the problem was developing ways to measure successful outcomes. In some criteria, such as applying math, science, and engineering, straightforward, objective means, such as testing the students’ competence, could reveal whether the programs were successful. Other criteria are more subjective and difficult to measure, such as the ability to function as a team member.

For example, to measure a person’s ability to function as a member of a team, students could evaluate themselves and each of their teammates on attributes that reflect what it means to contribute to a team, such as the ability to listen attentively to others without interrupting.

Using the multi-source assessment and feedback model to measure the outcomes is valuable because it doesn’t rely on a single measure, especially important with difficult to measure outcomes.

This multi-source assessment and feedback had another advantage: the outcomes for students involved in such surveys was better than for those who did not participate.

“Assessment processes not only provide valuable data on learning outcomes, but also have an impact on learning itself,” Besterfield-Sacre said. “Introducing a formal assessment process helps reinforce the learning objectives established for a specific course.”

—John Fedele