It’s June and these days we’re also busy marking exams. We’re also busy wondering why we give our students exams and what use they are (the exams, not the students!). What use assessment is, in fact. I have just entered the final marks for the course I have taught this semester and they are exactly the same marks I would have awarded each student one week after meeting them. Funnily, their marks did not depend on just a final exam but on four different items, with their corresponding percentages, etc, all that requiring Excel to be worked out… I don’t know what this says: that assessment only validates subjective impressions, that assessment that does not rate the exercises but the person, that I am such an experienced teacher that I know at first sight how students will perform (ehem!), that I could have saved myself a lot of hard work… Take your pick.

This was an elective course and I always prefer for assessing this type of course a paper rather than exams. This time, however, I decided to use exams for assessment, apart from a short essay written at home and a class presentation. I hated exams as a student and do not particularly like them as a teacher. One of my colleagues claims that we should never ask students to write papers, for they plagiarize all the time–which is an exaggeration… though also a constant fact in teachers’ lives. My position is quite the opposite: I do not see any equivalent situation in real life in which people would have to write a piece of academic work in a tightly limited time. I associate this, rather, with journalism and newspaper’s daily deadlines. Otherwise, why would anyone produce a piece on Wordsworth’s poetry, to name the first case that comes to my mind, in a very short time? It’s simply ridiculous. I’m rather of the persuasion, then, that exams only measure people’s ability to take exams. Or negative ability–I always performed well but only after bouts of nausea and vomiting that did nothing for my faith in the use of exams. I must have passed the last one in my doctoral days (I’m not counting vivas or oral examinations for tenure) and that surely was a happy day.

Accordingly, I play all kinds of tricks if I can manage to try to deconstruct exams. I believe that good academic work requires a reasonable time of preparation (not just of cramming) and I’m known to have given my students the exam questions in advance. I don’t care very much for failing students and I find that the students who fail in my courses usually trip themselves up by not handing in exercises or not taking the exams. If I get the chance, however, to help my students to do well enough for me to pass them, I’m happy. This is not the same as saying that no matter how they perform I’ll pass them, but rather that I don’t want anyone vomiting before taking one of my exams. I just want them to have studied and, above all, to have planned their exam question at home. I found out, however, that students given the exam questions in advance got quite nervous for a reason I failed to anticipate: if you know the questions in advance then a good deal of the justification to fail vanishes. Who would have thought that Prof. Martín would ask such a devious question? That seems to be the kind of thinking that comforts students that do poorly. Now, if Prof. Martín puts her questions in your hands, thus eliminating the surprise factor (not necessarily the deviousness), that’s another matter. Your inability to plan the answers is highlighted, which is, let’s be honest, much more embarrassing that simply being unable to answer a question you could never have anticipated.

You might argue that surprise is the whole point of exams and the target that collective groan you can hear when students find the questions too hard. This just happens to be a sound I do not enjoy (my exams, in contrast, seem to be the source of much sighing…). Anyway, what happened when I gave students an exam to take home, consider and plan is that still a few failed. I certainly felt less responsible for their failing, if you know what I mean. What I noticed was that the effort done in the actual writing in class was similar: the same stream of sighs, the same flushed faces and always the lack of time (some students would run out of time even if given five hours instead of two, it seems). The pressure had eased, the quality increased, hopefully there had been no vomiting, but, then, it was still an exam written by Prof. Martín.

For my latest course, I have tried another tactic: have the students write their own exam. In hindsight, I realize that no exam questions could ever match the deviousness of this proposal but let me say that I was not acting wickedly but in good faith. The group was small, only 15 students, and I explained that they should write a two-question exam using our habitual Department format: select a passage from the book we’ve studied (maximum 10 lines) and ask a question that can be developed in a short argumentative essay (maximum 500 words), referring both to the passage and to the book in question. It was clear to me that students would be very uncomfortable if I didn’t check their questions, so I gave myself the task of validating each exam a few days before the corresponding exam date. What I found is that students wrote, on the whole, perfectly valid questions just badly phrased. Some of the questions were simply too big in scope for a short essay but could mostly be re-used; others came multiplied by two or three (students seemed insecure about which version to use). None of the questions was insultingly easy to answer, and here’s where I noticed my own deviousness.

Imagine my students telling their peers in other courses ‘Sara has allowed us to write our own exam questions’. The answer from said peers should be ‘My!, you’re lucky, now you can’t fail!’. Now, most of my students probably replied at this point what one of them told me: ‘No way! This is the hardest thing I’ve done in my life!’ Why? Because they quickly realized that my proposal to let them write their own exams would not result in easier exams–no way I would validate shallow questions. Therefore, they had a twofold task: produce the kind of exam I would write myself and do it so that they could secure a pass. Lacking feedback from them (I have asked, and I’m waiting) I can only surmise that they told themselves: ‘Ok, so I need to make things as easy as possible for me while writing the exam as if I were a teacher’, demanding Sara Martín, in particular. Sure–only I had not gone that far in my own thinking when I proposed the experiment.

Exam one went well: nobody failed, though I believe that nobody performed either at a higher level than if I had written the exam myself. My guess (I need the feedback) is that students were more relaxed and confident about what they were doing, having got the annoying surprise element out of the equation. My colleagues say that written exams have the added bonus of offering exact information about the actual command of English each student has. Maybe. By giving students the questions or asking them to write their own questions, I also expect them to work on their English at home and produce far more polished exams (much easier to correct for me, too!). I don’t know how they do this: if it were up to me, I would write the answers at home (using the dictionary, etc, etc), try to memorize as much as I could and then write them in class. It might well be, however, that they have memorized outlines, I don’t know. I’m sure, though, that many language doubts and errors could be ironed out at home. This is fine by me for in this way they had to learn some English language in order to prepare the exam, in addition to the English Literature.

For the second exam I asked students to produce questions combining both a passage from the primary source and from a secondary source. I’m not sure whether this was my fault or not, but it seems that my instructions were a bit ambiguous about which secondary source to use. I had asked students to read one article for each novel and I expected they would use the ones I had selected for them. However, some students just chose other articles, which was a bit complicated to negotiate. I validated their exams eventually. What I found, and this was both silly and funny, was that the most complicated thing to do was validating the exams in which my own article was quoted. I sensed a kind of mutual embarrassment: students seemed to feel a bit awkward writing ‘As Martín claims’, which was not the case when they wrote ‘As Vint claims’, or ‘As Frelik claims’, for they didn’t know these academics personally. On my side, I found myself disagreeing with how students read my own article, even though their questions were perfectly valid. It felt very, very strange to be ‘Martín’ rather than ‘Sara’, as my students call me in our informal Department.

In both exams the contents reflected very accurately was what being discussed in class. The questions did refer with no significant exceptions to the issues we had discussed together though the passages chosen were not necessarily the ones I had selected for class discussion. The exams were, in short, more personal and less ‘parasitical’ of class discussion than I expected (this was my main fear). Some exams, particularly in the second series, were actually quite sophisticated. When marking them, I often marvelled that students who knew nothing about SF a few months back were confidently discussing artificial intelligence, genetic manipulation or post-humanism. Happy, then, as far as I’m concerned.

I failed, however, in just one thing. I decided to use exams because I wanted students to read the five novels in the course–if they wrote a paper, then they might read just one or two. What I failed to notice is that the second exam, covering three novels, should be much longer, perhaps two hours and a half, rather than one and a half. And we simply don’t have that kind of time. In old times exams had a separate schedule, apart from teaching time. One of my own teachers was famous for giving us exams that could run for four hours or more. Since 2009, however, exams are part of our teaching time, which means that the more exams you introduce the less time you have for actual teaching; also that they need to fit our 90-minute slots. Either I introduced a third exam, or I let students choose two of the three novels for the second exam, which is what I finally did.

Was the experiment worth carrying out, then? Certainly. I think that the class size was ideal, as validating the questions–not an easy task!–could be done in a reasonably short time. Also, these were fourth year students. I don’t see myself repeating the experiment with second-year students in my Victorian Literature course because a) they would panic, b) at about 50, the group s too big and validating the exams would consume too much time and energy. I believe that my experiment in tailor-made examination shows that asking everyone the same question is a bit of an absurdity for each student is motivated by different things in the same book. What we do when we ask all students in our class the same question, then, is just something convenient. We tend to ignore the fact that, beyond what each student has studied in preparation for the exam, some will automatically do well (or badly) because of the nature of the particular questions. I’m talking about Literature, not mathematics, of course…

Now, I would be really glad to get feedback from my students…

Comments are very welcome! (Thanks!) Just remember that I check them for spam; it might take a few days for yours to be available. Follow on Twitter the blog updates: @SaraMartinUAB. You may download the yearly volumes from http://ddd.uab.cat/record/116328. See my publications and activities on my personal web http://gent.uab.cat/saramartinalegre/