The issues:
- z-scores:
The use of z-scores in evaluating instructor performance has been shown to not be representative of teaching ability or performance. In fact, it is hard to determine any information about the instructor. The paper of P. Stark and R. Freishtat [1] comments that these Student Evaluation of Teaching (SET) do not measure teaching effectiveness and summary items like "overall effectiveness" seem most influnced by irrelevant factors such as gender, ethnicity, and attractiveness of the instructor; further, low response rates make it impossible to generalize reliably to the whole class. In fact, the category of "teacher effectiveness" can be predicted by a 30 second soundless video of the lecture. With pay tied to this z-score value, this is a critical issue for many TAs.
An idea is to form a teaching portfolio that includes items from student evaluations, sample quizzes, sample lectures, and feedback from observations. Not only would this help in forming a more comprehensive evaluation, but it would assist in future job applications for instructor positions.
Update: The department has indicated that unfortunately they are not moving past z-scores, but they do take other factors into consideration as well. Hopefully we can bring more clarity to this in the future.
- TA Improvements:
The issue of z-scores has led to a situation where some TAs with low z-score values have a significant disadvantage. That is, if their z-scores are too low, they are no longer assigned TA positions where they could be evaluated by students to improve their z-scores. While we still have the z-score system in place, I believe we should have an alternate system that allows these TAs to receive training to continue to improve—that is, in fact, what getting a PhD is all about. I propose that an alternate system allow TAs with low z-scores to participate in a sort of shadowing scenario where they can co-teach lessons in another TA’s course while the students observe. The students can then give the shadowing TA feedback via a paper or online method similar to the mid-semester evaluations. That way, these TAs can continue to build a teaching portfolio that they can submit for consideration for teaching assignments in the following semester.
Along with this idea of shadowing a TA, we ask which TAs or faculty would be willing to participate in observing classrooms upon request from the TA. We already ask which TAs would be open to having another TA observe their classroom, so this wouldn't be much more than an extension of that.
Update: The department is trialing a peer mentoring programs for TAs with classroom observations Fall 2016.
- Online Exams:
With some courses moving towards online exams, I have some concerns about the viability and validity of the results compared to current paper exams. With the current paper exams, we assume that they provide an accurate representation of the student's knowledge of the material, but even slight variations in the problems can alter the results. For normal exams, we usual use two reordered versions. In each version when the scores are compared, the averages can typically vary by 3-5% for each version (based upon my calculation for a few exams where I had this information). The issue is that the current setup breaks up the problems into certain categories in which each collection has numerous problem from which a subset of problems are randomly selected to be evaluated. Does this have the same variation on results? Can one student get a substantially harder or easier exam than another? Do the results of the exam provide an effective measure (or at lease comparable to our current paper exams) of the students of their knowledge? I hope that coming up with the answers can improve any potential use of the online examination system.
Update: The department will be discussing this topic to determine the quality of online exams in response to these issues.
- Office Hours:
Update: Spring 2016. The department is doing construction on the 2nd floor of the Math building during the Summer of 2016. During which they are reorganizing the advising office space and combining the two rooms for office hours into one larger room. This will hopefully ease all the concerns about the office hour space.