Assessment Rocks

The Four Horsemen of the Assessmentocalypse

Conquest - Famine - War - Death

The Four Horsemen of the Apocalypse

And when I saw the Dean put me in charge of assessment for my program, I heard the noises of much groaning and grumbling from thine own mouth. And thus I cried with a loud voice, "Why dost thou place upon me thy heaviest of thy burdens? How dost thy expect adequate completion of thine own many labors with these four terrible beasts placed before me?"

And the Dean, having mumbled something about "service," vanished amongst a whirl of accreditation paperwork.

And thus began my descent into the world of assessment, I began to plot how to defeat these four terrifying beasts:
1. Educational Purposes
2. Learning experiences
3. Organization of learning experiences
4. Assessing learning.

Follow my descent into the assessmentocalypse and eventual conquest over the beast of assessment in the sections below.

1. Slaying the Beast of Vague Educational Purposes

The Beast of the Vague Educational Purposes is a tough one to slay. Being vague is easy and convenient, for many reasons.

What, you expect me to describe those reasons?

Maybe I will...maybe I won't....

To determine the educational purpose the program will achieve, there are five important considerations:

Once you answer these questions, your next task is to write these out as learning outcomes statements. Some people (and some accreditors) insist these statements be written a certain way - starting with an action verb, and blah blah blah blah don't care and go away. My advice is to just write these statements in a way that is easy to understand and is clear to those who read them and use them (primarily the faculty in the program, the students in the program, and an occasional administrator - although if they really were passionate about this work then why did they assign it to me...big mistake on their part...).

2. Slaying the Beast of the Distasteful Learning Experiences

Once the learning outcoems are established, the next challenge is to identify the specific learning experiences that will produce the desired learning. This is usually done by identifying the low-enrollment courses that could benefit from a few more students and that are offered between 10-2 on Tuesday or Thursday. It also helps if the title of the course is somehow vaguely related to the desired learning, but that's just a nice bonus.

I warned you I can be a bit cynical!

But too often this is not far off from how the learning experiences are selected - there is often a focus on existing courses that can conveniently be dropped into the program's requirement list.

What really needs to be done is a deep dive into each course's learning experiences, and careful consideration of learning experiences that are to occur outside the classroom (such as an internship, practica, etc.).

For example, if you want to teach someone how to shred on their axe, you can't spend the entire semester asking students to read about various guitar designs and watching a documentary about Slash...

(To be honest, watching a documentary about Slash might be enough to make a student want to give up on Rock and Roll altogether.)

It is important to remember that students have lots of choices for majors, and if the experiences you offer don't help them progress in their learning toward what they want to do, then they'll go elsewhere. (They might even join...barf...a country-western band!)

3. Slaying the Beast of Disorganized Learning Experiences

"Two courses from group A, two courses froup group B, and three from group C!"

Slaying the beast of distatesful learning experiences isn't enough. Those high-quality, engaging, powerful learning experiences that you selected for your program must be organized in some way. Too often our "organization" looks like a lunch menu from Denny's and has similar results: you eat and eat, are shocked by the final bill, and leave feeling unsatisifed and a bit dirty.

There are two important principles for organization of learning experiences. The first is horizontal alignment. Horizontal alignment is about how learning experiences offered during the same semester support and reinforce each other. For example, if an accounting student is learning to use Excel to perform calculations in a macro economics class (e.g., "supply and demand"), then a horizontally-aligned curriculum would have that student also using Excel to perform calculations in their math class, in their finance class, and in their History of Medieval Mating Rituals of Manticores class (elective). Without horizontal alignment students see learning as siloed and struggle to integrate learning across the various learning experiences.

The second principle is vertical alignment.Vertical alignment is about how learning experiences are aligned over time - from semester to semester, from year to year. A vertically aligned curriculum will build learning intentionally over time, with learning experiences later in the program building upon the learning that occurred earlier in the program. It requires the intentional, systematic collaboration between those leading the learning experiences and careful guidance and direction to those enrolling in the learning experiences. Sometimes programs reject vertical alignment in favor of student (and program) convenience - the flexibility to take (or offer) a course at any time, in any order, makes progression through a program much easier. But think about it. Consider any topic you've learned in your life. Let's take learning to play the bass guitar solo in Led Zepplin's song Orion. If we break apart what is needed to play that solo into three parts - holding the bass; knowing the song; playing the song - it wouldn't make any sense at all to have someone attempt to learn to play the song without first knowing how to hold the bass or even being aware that Orion is a song by Metallica, not Led Zepplin. Flexibility and convenience are important, and should be considered. But don't sacrifice good learning for convenience - treat your program as if learning matters most!

I don't want to make this section too long, but it is one of my pet peeves. People come to me and say, "Assessment is so hard! What can we do to make it easier?" Almost always the problem isn't with assessment itself (collecting, analyzing, using data) - it's almost always a problem with a poorly designed, disorganized, unintentional program that cares more about accumulating seat time and credit hour generation than about supporting transformative student learning. So if you are struggling with assessment, take a step back and spend some time on your program's design.

4. Slaying the Beast of the Forgotten Assessment Process

We all have a limited time on this Earth. Do you want to waste that limited time doing things poorly? Especially when you still have time to learn, to improve, and to make those things better?

We can't all be Tony Iommi, but we can all be better than what we are now.

This is what assessment is all about.

The Four Horsemen of the Apocalypse

Assessment is a cycle of learning and improvement. The Plan - Do - Study - Act cycle has been tested and used by thousands of organizations all around the world to engage in continuous improvement and organizational learning. But enough jargon mumbo jumbo. Here's what you do:

PLAN: Identify a goal or a purpose - this could be based on a previous cycle, on anecdotal observations from faculty, or focused on your learning outcomes. It works best if it is something people care about.

DO: Carry out your plan. Return to the plan periodically (say, every month) to ensure you are on-track.

Study: Monitor your progress and evidence of achievement by collecting and analyzing assessment data. Use your study to LEARN - what worked, what did not, what should you keep doing or do differently next time?

ACT: Make use of what you learned by sharing it with others and implementing changes. Feed your actions into the next PDSA cycle.

Ascent

And lo, upon the Dean's return, the Dean hath discovered a transformed program, unrecognizable to even thine own eyes, with engaged faculty, happy, learning students, and growing enrollment. And the Dean rewardeth me with a continuation of thy assessment assignment, to which there was great cheering and celebration, even from thine own mouth.

And so, from this day forward, I shall dwell in the house of assessment, forever.