No Apple for Skandera: Teachers tell her why she’s wrong, wrong, wrong
By Peter Moulson, Special Correspondent
When the conservative Albuquerque Journal reports that “a nonpartisan group of scientists and mathematicians says New Mexico’s new A-F school grading system is too complex for most people to understand, including principals and superintendents”, one has to pause.
According to the syndicated article “The group, called the Coalition for Excellence in Science and Math Education, is a nonprofit with several hundred members. It has been active in New Mexico since 1997, and its mission is to “improve science education and science literacy for all citizens,” according to its website.
Adding oranges and cows to derive pickup trucks
“The coalition’s report also questioned whether it is mathematically appropriate to add different measures together. Specifically, the A-F grades use simple measures of how many students are scoring at the “proficient” level, which are added to measures of how much students are improving. The “improvement” scores control for demographic factors, like poverty.
“The coalition contends it is mathematically inappropriate to add these measures together. According to the report, such addition is “something like adding oranges and cows to derive pickup trucks. The result is not obviously meaningful.”
“The group also believes the system adds elements together that aren’t compatible, and that the formula’s sensitivity to small changes results in unreasonable grade changes from one year to the next.”
Teachers have a say
Between the packed Santa Fe hearing room and the demonstrators outside, around 500 teachers gathered on July 18th to make their anger known about the non-legislative Administrative Rule that is intended to tie their pay to the A-F scale of their schools, whether they are individually graded as teachers or not.
Even though New Mexico lawmakers declined to pass legislation to allow the Secretary-designate the power to tie teacher salaries to the performance of their schools, it looks likely to go ahead through the use of an administrative “rule” by the Martinez administration.
Everyone agrees that teachers must be accountable for their performance
What is wrong with making teachers accountable? Almost no one in the state argues against some sort of change to teacher evaluation. However, is the Standards Based Assessment any better for teachers than they are for students?
The July 18 hearing featured Secretary-designate Hanna Skandera (at a salary of $125,000), accompanied by Matthew Montaño, director of educator quality ($93,000), and their PED attorney Willie Brown ($95,000), with brief statements made by interested parties, but no comment was made by the Secretary-designate.
Skandera appeared to listen attentively, taking notes and smiling as impassioned teachers made their points about the risks and follies of tying 35% of the salaries to the Standards Based Assessment–the testing model that is detested by teachers (“we can’t teach during the tests, we have to teach to the test instead of educating our students”); researchers (“this means a middle school or high school social studies, music, technology, or PE teacher’s “effectiveness”–and therefore pay–would depend not on their own work, but on the work of the [tested] math and language arts teachers”); and students (“I can just about stand 2 out of 8 days of tests, but after that I doodle on the papers and fill in the bubbles at random”.)
Putting the test to the test
One of the few people in the packed room with a real expert understanding of the validity of the testing that affects our children, and may dominate our teachers’ lives, is herself a highly-qualified teacher. Chelsea Ketchum teaches middle school math, science, and an engineering elective at Tony Hillerman Middle School in APS. She also holds a Master’s Degree of Education, Curriculum and Instruction (including an Assessment Validity course.)
Assessment Validity Expert Chelsea Ketchum says “NO!”
Ketchum ‘s statements about the failure of the SBA tests themselves managed to raise looks of astonishment from the Secretary-designate.
“I earned my Master’s Degree in 2008 in Educational Curriculum and Instruction. After examining the Teacher and School Effectiveness Rule proposed by the New Mexico Public Education Department I have found two major reasons to not support this document as law.
“In section 8.8 of the rule, Effectiveness Evaluation Systems (EES), 50% of a teacher’s effectiveness evaluation is based on measures of student growth through a combination of 35% SBA Data and 15% assessments that are selected by school districts. Section 8.9 of the rule, “Student Achievement Growth and Student Assessment Programs,” gives two options for how teachers who do not teach grades or subjects with SBA tests will be evaluated.
“The first way in letter D 2 states “Student growth shall be measured by… the school’s A through F letter grade… for courses in which enrolled students do not take the state assessment.” This means teachers in a middle school or high school social studies, music, technology, PE or any elective teacher’s effectiveness (and pay) would depend not on their own work, but on the work of the math and language arts teachers.
“The second way in letter E reads ‘Beginning with the 2013-2014 school year, each district shall be responsible for measuring the achievement gains of students in all subjects and grade levels.’ This means every student in every grade K-12 and in every course in middle and high school will take a high-stakes test to determine their teacher’s effectiveness. To measure teacher effectiveness in electives courses, letter F of section 8.9 gives the option that, ‘A school district may develop its own assessment that measures student achievement growth for classroom teachers who do not teach in a standards-based assessment grade or subject.’
District-developed tests are neither valid nor reliable
“My problem with this is district-developed tests are neither valid nor reliable and cannot give a valid or reliable assessment of teacher effectiveness. In my master’s degree coursework I took a class on how to make a valid assessment. In order for an assessment to be valid, after its creation it must first be aligned to the course content by an expert panel
review board, second it must be examined for errors, third it must be field tested with a large number of students to eliminate questions biased towards or against certain groups, and finally it must be sent back to the panel review board with data from the test group for final editing.
“Examples of valid tests are the ACT and SAT. I have seen what happens when a non-valid test is used for high-stakes testing. Three times a year in Albuquerque Public Schools students are given ‘District Benchmark Assessment’ (DBA) exams which are used to decide which students are placed in summer school in grades 3-8. On every test I gave, my best and brightest students would come to me pointing out errors in the test such as no right answer, multiple right answers, and unanswerable questions based on the information given.
Show a movie and give no homework: Grade A+ for that teacher
“My second major reason to not support the teacher effectiveness rule as law is non-educators, in the document called multiple sources, are being used to determine 25% of a teacher’s effectiveness and pay. In section 8.8 of the rule, Effectiveness Evaluation Systems, letter D3 reads, ‘Multiple sources may include giving parents and students opportunities to provide input into effectiveness evaluations.’ I appreciate both parent and student feedback
“But this doesn’t mean a child age five or age 18 should decide how effective I am, because students have not been trained on what an effective teacher looks like, sounds like, or does. Students may highly rate teachers they think are their friend, or teachers who show movies all day and give everyone and A as highly effective and the teacher who gives homework, uses new technology, and gives students the grade they earn as ineffective.
“Most teenage students and parents of teens will rate the effectiveness of teachers based on the grade they receive without consideration of how the student’s efforts affected their grade. Parents of secondary students have very limited interaction with all of their student’s teachers. These parents may see a teacher perhaps two times a year at conferences, get a weekly email listing announcements, and often only get a phone call if their student is in trouble due to the fact that most secondary teachers see 160 or more students per day.
I am not afraid of being evaluated or assessed
“Parents also are not trained on what an effective teacher entails. If multiple measures are for parents and students to evaluate 25% of a teacher’s pay, this will take away a teacher’s right to give students the grades they earn and make them susceptible make them susceptible to the whims of all parents, no matter how demanding or unreasonable. Multiple measures should instead be based on teacher leadership, participation in committees, and running academic extracurricular activities.
“As it is currently written the Teacher Effectiveness rule would do more harm than the good it intends. I am not afraid of being evaluated or assessed as long as it is done in a just way with valid data and by evaluators trained in educational practices. ”
Is the Florida model a success?
Before coming to New Mexico, Hanna Skandera spearheaded the introduction of the A-F assessment scheme in Florida. We’ll need to keep an eye on how things go down there if the Martinez administration insists on duplicating it here.
# # #