Aug 01 2012

How to evaluate teachers

Note:  A publisher has expressed interest in having me do a book about my finding on the ed reform movement, including what I think productive reform would look like.  I’m going to write some stream-of-consciousness drafts and publish them here on the blog, since it motivates me to write when I know it will get instantly read by some people.

—————————————————————————————————————–

I’ve been criticized by some for only writing about what I’m against, and not enough about what I’m for.

There is a good reason for this which I’ll use an analogy to explain:  If you have a house and you’d like to remodel the bathroom, but then someone sets your house on fire — you’ve got to direct all your efforts toward putting the fire out before you can concentrate again on remodeling the bathroom.

That’s how I see ed reform right now.  There are destructive policies getting implemented all over the country.  They are spreading like a disease, and I see it as my job to stop the outbreak.

But, if there were no fire, I’d be advocating for the kinds of improvements that I think would help education in this country.  After all, it is in my nature to complain about things, and I certainly would still have a lot to complain about.

For example, I think that math education in this country is completely ‘broken.’  Somewhere along the line math turned into a mindless exercise that is about as useful as learning to speak Klingon.  I’d be interested in working on ways to restore math back to the type of things that made Plato make it one of the main subjects of study way back when.

Another thing that I would want to tackle is teacher evaluation.  The way it is now is way too neutral.  Still, neutral is better than destructive, but it still is pretty neutral.

Though I have not been a principal, I have supervised many student teachers and in that supervision I have done observations while they taught.  I have also seen a lot of teachers in my own school teach as I patrol the halls for a few periods every day as a part-time dean.  Since I have a lot of classroom experience, I can gauge how good a teacher is in about two minutes of observation.  When a teacher is ‘good,’ there is a tangible energy in the room.  If I had to quantify some of what that energy is, I could easily make up some metrics:  How many students volunteer, on average, each time a question is asked in class?  How much ‘wait-time’ does the teacher use after asking a question?  What percent of students actively participate in a period?  What percent of time in the class is idle?  What percent of time is a teacher ‘lecturing’?  How efficient does the teacher go over homework?  These are things that I, as someone with a lot of experience, doesn’t really ‘need’ to calculate for someone who is doing a good job, but for someone who is faltering, these are some metrics that I could use to help this teacher improve.

If I saw a teacher I felt was ‘good’ based on my observations, I would not really need to see her value-added scores.  If a computer told me that a teacher I knew was good was actually bad, I’d ignore it just as I would if a computer told me that Marilyn Monroe was not attractive because of her 10 dress size.  I would also ignore a computer that told me that a teacher I knew to be ‘poor’ was actually good because of value-added computations.

One issue we have today with teacher evaluations is that many principals are not qualified to evaluate teachers through quick informal observations.  Some principals have never even taught before.  Some have only taught for two years.  I think this is one of the reasons that people are pushing so much for value-added nowadays.  A principal who does not have the ability to get a feel for the quality of a teacher, just by sitting in the class for a few minutes, is more likely to misjudge if a teacher is good or bad.  So principal evaluations are the main thing we need — but only if the principals have the ability to do this, which, unfortunately, many don’t.

Another aspect of teacher evaluation is the teacher’s self-evaluation.  Though I find it very easy to go into a classroom and quickly point out a teachers strengths and weaknesses, I often think it is ironic that when I am teaching, I often make the same sorts of mistakes that I notice in other teachers.  For instance, I often call on just the students who are raising their hands.  When I see someone else do this, I think ‘they should be spreading it out a little,’ but for some reason I do this too much since ‘in the moment’ it is hard to cold call on someone when you are low on time and know that it could jeopardize your flow and you may not complete the lesson.

But if you showed me a video of myself teaching, I could easily critique myself.  And if the stakes were low, I would gladly make a list of areas I could improve upon.  So video taped lessons with self-assessment is something that I would approve of.

Often, today, we hear a lot about how ‘student achievement’ has not been factored into teacher evaluation in the past which is why value-added is a step in the right direction, even if it is not perfect.  A favorite expression of Arne Duncan’s is “We can’t let the perfect be the enemy of the good,” a quote from Voltaire.  So because value-added is merely ‘good’ and not ‘perfect’ we shouldn’t abandon it.  But who says that it is ‘good’?  Right now I’d say it is awful, and I don’t think Voltaire would agree “We can’t let the perfect be the enemy of the awful.”  Incorporating value-added into an evaluation is not the same, I think, as incorporating ‘student achievement.’  If it didn’t have ‘student achievement’ in it before, then it still doesn’t have it.

Now ‘student achievement’ is something that is tough to measure directly.  But that does not mean that ‘evidence of student achievement’ which is also an expression I see a lot, can’t be measured.  If I were evaluating a chef, would it be possible if I did not (or even could not) taste the food.  I think so.  If a professional chef watched me as I made my famous cream-of-mushroom mac and cheese, I’m sure that the chef would be able to evaluate my abilities without tasting it.

If a teacher comes on time every day, has a good lesson, and teaches the lesson in an interactive way, allowing time for the students to practice, and then the teacher assesses the class informally and formally, then the students will progress.  There is just no way that all that stuff can be going on and the kids are not learning.

Now if you really want more concrete evidence of student achievement, it still can only be done indirectly, though maybe a bit closer to directly.  One way to do this would be for teachers to issue a pretest on the topic about to be taught and then a posttest afterwards.  This is different than the value-added growth.  In value-added they try to figure out based on what kids got on the 4th grade test, what they should get on the 5th grade test with an ‘average’ teacher.  To see what the kids actually learned, though, it would be better to give them the 5th grade test in the beginning of the school year (most kids would get nearly a 0%) and then give them the exact same test at the end of 5th grade.  Even if a kid failed with a 40%, that would still be 40% of stuff the kid learned.  Isn’t this what we do when we learn a musical instrument.  First we try to play the piece on the piano and then when we’ve completed our practice we try to play the exact same piece, only better.

Giving a pretest is good practice for a teacher anyway.  It is not something I do as much as I should because it takes time away from teaching, but if it was required that I do that for a certain percent of my assessments, I’d be willing to do that for the sake of an evaluation so a principal can see how much my students really learned based on the pretest / posttest comparison.

The better prepared I am, the better my lessons generally go.  If there were some kind of easy way for me to submit my lessons for my principal to look at, and also a way for me then to share that lesson with my co-workers, then that would be a system that could be incorporated into my evaluation and also into the collaborative process.

I don’t know how other teachers would feel about this, but I could see a scenario where I wouldn’t object to much of, if not my entire, year being video taped.  I know this is an invasion of privacy, but I think it would make me teach a bit better.  Certainly I’d teach a little differently, though not necessarily better.  The benefit to me would be that maybe those videos can also be used as ‘evidence’ when a student misbehaves.  It would be interesting to be able to show a parent the video and then see if that parent still doesn’t believe that her kids threw a pencil at another kid.

What I don’t like about the video is that I might be too self conscious and might not joke around as much.  I don’t joke around that much anyway, but there are times where I might spend a few minutes going off on a non-math tangent.  This isn’t a bad thing to do if it is done in moderation.  But what if someone were to make a montage of all my off-subject ‘riffs’?  Well it could easily be used against me.  Still, if someone wanted to watch my entire year and the few minutes of riffs were put into context that way, I would not be ashamed that I ‘wasted’ a few minutes as part of making my class a human experience.

Anyway, those are just some of my thoughts about teacher evaluation.  Really anything is better than value-added, but that doesn’t mean that evaluations can’t be made more useful for helping teachers improve.

13 Responses

  1. I think you’re definitely headed in the right direction. I agreed with most everything you said in my post here http://developingprofessionalstaff-mpls.blogspot.com/2012/04/evaluating-teachers-part-four.html

    We have all the technology to do what you’ve envisioned; it will take a bit more consensus building, though, before that vision takes shape.

  2. Michael Fiorillo

    Well said, and I also agree with most of what you say, especially the part about a skilled eye being able to recognize good teaching in a few minutes (which of course undermines much of the hype about how “the teacher evaluation system is broken and must be replaced by…” Now insert faddish, politically-juiced false panacea into the blank).

    I strongly disagree with you about the lesson plan submission and videotaping, though.

    While I assume this is only something you’d “permit” after the current testing mania/teacher scapegoating/smash-and-grab regime has gone into the Circular File of history, I’d still object, for the following reasons:

    - While I am happy to work collaboratively with my colleagues, sharing materials and methods, pacing, flow, student comprehension and retention, serendipity and useful digressions (dare I admit to them?) are different for each class, and I don’t think Principals should be given any more tools to micromanage than they already have. This is especially the case now that so many Principals and APs have less classroom experience than the people they are supervising.

    - As for videotaping: creepy, with an ever present potential for abuse of all kinds. You mention caveats to it, but I’m not comfortable with an eye-in-the-sky on me and my students under any circumstances.

    Teaching should be a safe, engaging, full-on human encounter. It’s face-to-face, with everything that means. That’s what our students deserve from us – among many other things, but this is fundamental – and what we likewise deserve from those who would observe us.

    My classroom door is always open, so if you want to observe my class, come on in. Come in on consecutive days, if you like: supervisors certainly have that authority. But we show our faces in my classroom, literally and figuratively, and if you’re observing, you should too.

    In fact, please feel welcome to participate in the class. Last year my Principal participated extensively in a class he came to formally observe, and it was a good experience for all. That might be a tiny exception in the overall scheme most teachers work in, but it can happen and it works for everyone when it does.

    Maybe it could even develop in an organic sort of way, and be replicated widely, if public school teachers had a respite from all the scapegoating and attacks against them and their schools.

  3. Dave W

    Instruction continues to be driven by administration, which in my opinion, is one of the biggest problems in education today. It takes a secure, experienced, and strong school leader to step back and realize that their experience in the classroom is usually very limited and that they should spend more time supporting their teachers and holding the ship together. That job is difficult enough for 180 days a year. If administrators actually spent the majority of their day operating the building as efficiently as possible, handling logistics, discipline, and keeping their teacher’s happy, schools would be much better off. Unfortunately they spend a large portion of their days in useless meetings trying to find ways to raise tests scores.

    My experience in majority Title 1 high schools suggests otherwise, although I would imagine in larger schools the principals and VP’s probably don’t spend as much time in classrooms, which is probably a good thing. I have too often witnessed poor or useless instructional advice from these administrators who have actually hurt the development of our more inexperienced teachers. At least the experienced and skilled teachers are better at ignoring this advice and snickering to their colleagues about it afterwards.

    I have no problem with administrators stopping in my room to check things out and even take some time to learn something new once in a while, so they don’t get too disconnected from the struggle teachers face on a daily basis. Heck, I wouldn’t mind observing them to see how they execute their meetings and whether they do a good job of handling discipline directly with students :)

    Let me be clear, I’m not saying all administrators can’t provide decent feedback, because there are some school leaders who have had significant classroom and instructional coaching experience, but sadly I haven’t seen many in my nearly 10 years of teaching experience.

    This leads me to why curriculum specialists, instructional coaches, teachers, and department chairs should guide the evaluation and development process when it comes to instruction. Leadership in the building needs to relegate the instructional process to those who have the expertise. Leadership can hold their department heads, instructional coaches, and curriculum specialists accountable if they are not getting the results they want.

    Both teaching and administrating a building are challenging in their own right, and when we cross into each other’s domains we tend to screw things up and make the school a less efficient place.

    Therefore, videotaping, lesson planning, and collaborating with other teachers when it comes to the instructional process should be guided by the instructional experts. If this was the case, teachers would feel a lot more comfortable and supported, which would make our instructors better and more accountable for great teaching.

    Alluding to your “med-reform” post, can you imagine hospital administrators observing and critiquing a surgeon while they perform surgery? I doubt it happens, if ever.

    Let’s leave the evaluation of instruction up to the experts, not school administrators, policy makers, educational non-profits, or central office bureaucrats.

    My skepticism unfortunately tells me otherwise. “You don’t know, what you don’t know.”

    • I think those specialists and coaches need to get their hands dirty once in awhile, though. In my teaching career, I’ve watched coaches get more rigid and out of touch with the daily practice of teaching as they spend more and more years out of the classroom. They don’t coach as well, and improving classroom practice goes from being a shared endeavor to a student/pupil relationship.

      I think making sure coaches and specialists are themselves evaluated on the success of those they work with and their relationships with their teaching peers would help this a lot, though.

      • Dave W

        I concur, they should not be immune to getting their hands dirty and should actually co-teach, or demonstrate lessons in actual classrooms with real kids. Maybe there should be a rotation after a certain time period. Ultimately, this would be up to the school leader, but if I was in charge I would make sure my curriculum specialists and coaches were teacher’s first and spent some of their time teaching a class, at least one a year.

  4. Dufrense

    Dave, I agree. It’s imperative that observations be conducted by people with instructional expertise. From what I’ve read of Peer Assisted Review, it strikes me as just that sort of approach.

    Gary, I concur with pretty much all your ideas. I’m wary of year-long videotaping, though.

    Regarding value-added, you’re spot on. I wonder if Duncan genuine believes it’s “good.” Judging a teacher by value-added scores is about as useful as measuring a music artist’s quality by album sales.

  5. Steve M

    Just as poverty is the greatest obstacle in our efforts to overcome our country’s educational, health, economic and justice issues, I believe that 90% of reform efforts (and monies) should focus on K-8 students. Here we are, a bunch of high school teachers, harping away at what needs to be done when nearly all of the problems start when our students are younger.

    For example, a former colleague of mine worked in drug rehab for twenty years before he became a health instructor at a high school. As a teacher, he did a series of studies over a period of three years and found that the majority of our students that developed drug problems did so during the pivotal 7th grade. So, we had something to work with.

    Likewise, elementary school teachers usually inform me that the 3rd grade is a pivotal year for students. That is when they begin to acquire and internalize many destructive behaviors. Yet, my school district made it a policy long ago to make grade K-2 class sizes 22:1 and grade 3-5 class sizes 32:1.

    Meaningful reform needs to begin with identifying when and where major issues develop.

  6. Cal

    “Here we are, a bunch of high school teachers, harping away at what needs to be done when nearly all of the problems start when our students are younger.”

    This is exactly what reformers believe, and there’s no evidence to support this. Far more likely is that things go right in elementary school, and cognitive demands and adolescence knock things off course in high school. We’ve spent a decade or more focusing on elementary school.

    Gary, I would be very distrustful of any administrator who dismissed progress in test scores because he personally liked the teacher’s method, or vice versa. If the teacher’s students aren’t learning, then that’s a problem. If the teacher’s students are learning, then it’s not a problem (short of abuse or inappropriate behavior).

    It may be that the students aren’t able to show their knowledge on the tests, of course. But if the tests can gauge progress, then the tests should always trump an administrator. Otherwise you have a friggin gulag.

  7. Steve M

    “Far more likely is that things go right in elementary school, and cognitive demands and adolescence knock things off course in high school.”

    Cal, I am not an administrator and I’ve spent eighteen years teaching high school…fourteen of them in what used to be called “category 1-1″ schools. A “category 1″ school is one that ranks in the bottom decile of the schools in a state. A “category 1-1″ school ranks in the bottom ten percent of that decile [I agree, it was a stupid way to designate a school in the bottom 1% of the entire state...California in my case].

    I agree that adolescence can prove trying, but if you seriously think that even a small majority of issues arise after the age of fourteen and that it is the “cognitive demand” of high school that primarily knocks students off course, then you probably have not been teaching for any length of time and need to seriously immerse yourself in the issues at hand [I'm really reining in my desire to insult you here].

    Gary, I would very much recommend that you focus your efforts (if you’re really going to be writing such a book) on addressing the issues younger students face. Otherwise, you’re going to fall prey to critics who state that you don’t address the needs of the majority of students.

  8. Cal

    I’ve been teaching for a while, primarily in Title I schools, and anyone who doesn’t realize that the cognitive demands of high school are one of the single biggest reasons for “failure”–that is, not being able to learn advanced math or understand complex literature–doesn’t know
    what “cognitive” means.

    Reformers and progressives both have this magical belief in elementary school, when there’s no evidence that kids are falling off in those years. Test scores are higher, kids have better attitudes towards school, and so on. It’s in high school that kids fall off, because we demand that all kids, regardless of their cognitive ability, take advanced courses. This is pretty obvious, I’m well-versed in the issues at hand, and I could care less whether you insult me or not, because well-meaning teachers who believe in Tinkerbell aren’t high on my list of people with opinions that matter.

  9. Dufrense

    “It’s in high school that kids fall off, because we demand that all kids, regardless of their cognitive ability, take advanced courses.”

    This is one thing that concerns me with the assessments associated with the Common Core. What will be asked of all students is akin to what the AP Language & Comp test asks of my advanced students. I’ve yet to see any compelling evidence that every high school graduate needs to write analytically at that level.

  10. First Grade Teacher

    Rather than engage in an “either/or” debate I would urge Gary to think about primary education as a separate entity altogether. The pretest/lecture/practice/post-test standard approach used in high schools is not desirable, feasible, or possible in primary grades.

    We primary teachers are juggling many balls in the air at the same time: teaching children how to read, comprehend, write, think, compute, etc.; teaching children how to “do” school and be in a setting where their needs and wants are determined by a group rather than a parent or caregiver; teaching children how to self-regulate their behaviors and responses in always changing situations; and teaching children empathy, discernment, research skills, and evaluation skills in varying contexts.

    In the beginning of primary education you cannot use written or overly verbose instructions because the majority of students can’t read or understand what you are saying. You have to limit your use of educational jargon because they have no framework for understanding it (yet).

    The attention spans of young children are said to be equal to their age +/- a few more minutes; instruction must be delivered in short bursts of highly engaging activity with frequent changes and variety.

    Before you can assign children to work in a group or do a lab, exploration, or problem-solving activity you have to spend a great deal of time teaching them how to work that way with gradual release of teacher support.

    Rather than going on endlessly I will end by saying that primary education is highly specialized with very different requirements than teaching self-determined, literate students who already comprehend reading (at any success level), basic writing, and are accustomed to lecture/note-taking. Primary teaching requires a set of very specialized skills that cover a broad area of subjects and specialization to be effective.

    I read with great apprehension another blog on this site from a beginning TFA teacher who was assigned to a primary special needs class. Talk about throwing someone in the deep end who can’t swim!

    Evaluating primary teachers should look and be very, very different from evaluating a high school biology teacher and many administrators themselves lack this understanding and experience. Many of the problems that inevitably arise in this phase of education (and they do — children at this age don’t all develop at the same pace as dictated by standards lists and government requirements) could be eliminated or alleviated before the students get to middle and high school.

  11. Peter Meyer

    My two cents: I think any student would tell you that the best teachers are the ones who a) know and b) love the subject. And, as you say, you can guage that in about 2 minutes.  Let’s build better teachers — the evaluations, in a collaborative school, would take care of themselves. Here’s the Build a Better Teacher program: Get a great K–12 education, a college degree in one of the five basic subjects (history, math, literature, etc.), two years of a master program honing content expertise and learning the basics of classroom management, then one year  of on the job training….  Can’t wait to read the books.

Post a comment

About this Blog

By a somewhat frustrated 1991 alum

Region
Houston
Grade
High School
Subject
Math

Subscribe to this blog (feed)


Subscribe via RSS

”subscribe

Reluctant Disciplinarian on Amazon

Beyond Survival On Amazon

RSS Feed

Subscribe