A conversation on closed booked exams
Ever meet someone who’s so ridiculously smart it feels surreal? Not just “got straight A’s in high school” smart. I’m talking “Einstein and Curie having brunch, and their conversation somehow spawns a person” smart. The kind of person who’s always studying. You’ll say, “Hey, did you catch the new Spider-Man movie?” and they’re like, “Nah, I was busy prepping for the calculus exam by rereading Principia Mathematica.” So, naturally, you think, “Oh, they must’ve crushed the exam.” But nope — they go, “Nah, I totally bombed it. I’ll be lucky to pass.” And when you ask if they do anything besides study, they casually mention, “I paint.” Just as you start to feel relieved, they hit you with, “…histological stains of cancer cells for a paper I’m submitting to The Journal of Pathology.” And the whole time, you’re standing there thinking, How does someone like this even exist? And how quickly can I leave without them noticing?
To bring a point to this terribly long winded joke (if you want to call it that), is that these type of people don’t pursue mastery, they pursue comprehension and mastery ensues. The question that I’m asking in this article is “are closed book exams a good way of assessing mastery? and if not, what is?”
Previous Paradigm of Pedagogy
When you think of examinations, you typically think of sitting in either a classroom or a large gym at a table slowly filling out a bubble sheet. However, this was not the case for most of western educational history. In fact, for most the most past, written exams were not completely the most common form of examination. Originally, in western medieval times, obtaining a degree would depend on the student’s ability to impress their teacher, and perhaps then the student would be invited to audition for the chance to get a degree. It wasn’t until a cleric from Europe ventured to China to examine Chinese educational practice. In China, extremely difficult examinations were required for most governmental and civil positions; in order to pass those exams, academies were created to help “teach” students to pass those exams. The cleric was super impressed by the very competitive and merit-centric system that the Chinese had implemented and decided to bring it back to Europe. From there, written examinations have now blossomed into the atrocious mess that we have now come to love.
Let’s revisit the point about mastery and comprehension that I made earlier with my really bad joke, are these types of examinations testing mastery or comprehension? Well, I would say neither. Mastery suggests that you can take a topic and skillfully construct a meaningful application of it such that it actually functions in a unique and novel way. Whether that be using calculus to compute marginal profit rates or use your English skills to write a legislative document. Whatever it is, you should be able to apply it and use it well in different contexts. This is why I say you should pursue comprehension, as you can really only do that if you understand the topics very well. Which is also why I say these types of exams do not accomplish testing either of these things. Traditional forms of examinations (mainly multiple choice questions) ask that you regurgitate information that you’ve spent memorizing on a sheet of paper. Your ability to understand is irrelevant if all you’re being asked to do is select the correct option of four possible options. I mean what good is that? You’re giving the students the possible answers and not even giving them a chance to come up with it on their own. It’s like the academic equivalent of spoiling the movie. In fact, most students recognize that you don’t even really need to understand the material completely, rather, you just need to eliminate the options that make the least sense. How bizzare is that? You’ve created a system where understanding the content of an assessment can be circumvented with “test taking tactics” (which could honestly be called good guessing strategies). One could argue that this is an intentional part of the test, to deduce the most likely answer; but I would argue that this is terrible design since you’re effectively trying to trick the student instead of trying to assess their understanding. “But only students who know their stuff can’t be tricked!” Ok, this seems like you’re preying on students by setting up traps instead of giving them a chance to show they know their stuff. This is what I will call pernicious pedagogy, a form of teaching and assessing oriented on attacking the students understanding such that it’s detrimental to the overall learning of the student. I’m not going to be explore this idea too much since I think it deserves a more nuanced and thorough examination.
In the modern day setting, this way of assessing students has become archaic and I would even say detrimental. Let’s think about the purpose of education and why someone would even pursue higher education beyond primary school. Well, that’s obvious, to get a job. Er, maybe it’s actually to learn things? Or maybe apply the knowledge that I learn in school to the real world so that I can earn money? Is that different than trying to get a job? Look, there’s a lot of debate on what the real purpose of education is, and that’s not the point of this article. What I will say is that, for most people, we go to school to be able to get a job. So, our assessments should build our ability to do that right? Well, they don’t, at least that’s not how most assessments work. Traditional examinations focus on memory recall which is based on the idea that you should be able to remember a lot of things whenever you need it. For example, suppose you are taking an exam on history, some of the questions will be phrased as “When did X occur? Who did this important feat? What is Y called?”. Questions that don’t focus on actually using/applying the knowledge but regurgitating the facts given in a text. This is an outdated idea. The advent of the internet has revolutionized the accessibility of information. That is to say that information has become a commodity. Let me repeat that since I think it’s a particularly salient point, information is a commodity. There has been a shift from memory recall to information retrieval. Thus, examinations focused on asking students to regurgitate information onto a piece of paper are redundant since students could just as easily look this information up. I can not think of a scenario where you would be tasked with something important and your employer would prohibit the use of external material. Like, imagine if your employer said “Okay So-and-so, you have to write this super important document that is paramount to the success of this company. EXCEPT, you have to do it from memory”. What purpose does memorizing and regurgitating information serve? It’s the most banal and ineffective form of assessments.
Enter Online School and Open Book Exams
The current pandemic has left us in a weird situation school-wise, everything shifted to a virtual format and lectures were being hosted on Zoom in lieu of large auditoriums. This shift has also left a bigger question for teachers, how do we proctors exams? if at all? I’ll be honest, I don’t think relying on student’s integrity and trusting them not to cheat in closed-book exams is the best method. If cheating was not rampant before, it sure will be if exams go online. So teachers were forced to make the transition from closed book exams to open book exams. Oh boy, let me tell you how much of a good move that was. Student’s can’t cheat by looking things up or by referring to their notes if the questions on the exam already account for that. Teachers have to be clever in their assessment design since low-effort questions could easily be googled or looked up in the textbook. Not only that, they could no longer recycle questions they found in teaching manuals (I always hated this) since solutions to those are likely to be online. However, this also meant that teachers had to be more diligent in marking these assessments since there is usually no clear cut answer/method to do things. An unfortunate and tedious down side of a better mode of examination. But, the best thing was that memory recall questions were mostly eliminated. Instead, you had questions about the application of the material instead, which forced you to know your stuff and apply it rather than memorize everything. Thereby reinforcing one of the most important skills that I think a student could have: information retrieval.
Why is information retrieval one of the most important skills that a person can have? Recall that information is a commodity in the age of the internet, if you ever need to know factually correct information then you can just look it up. That’s incredible! The fact that you have an entire atlas of information containing almost everything that humanity has written is mind-boggling to me. So being able to memorize/store that information is useless, since a computer can store data better than any human. Hence, being able to retrieve the relevant information is a much more valuable skill set. Let me repeat, finding the correct information is super important. In software development, we like to joke that the entire job is just googling queries and copy/pasting code snippets from StackOverflow. That is the canonical example of information retrieval being extremely valuable; being able to find code snippets that is relevant to your software application among the sea of all possible coding snippets. That is the power and advantage of information retrieval, you’re never reinventing the wheel since someone will have likely already made a better wheel. Okay, but you might be saying that “this example is exclusive to engineering”. Fair, how about I give two examples in life sciences (Cell Biology) and the humanities (History).
A Case Study: Cell Biology
Conventional study of biology and biological phenomena has mainly been done on classification and recall of that classification. In fact, Aristotle founded his theory of biology in classification of species which many regard to be a significant contribution to the field. However, since then we have learned an incredible amount about the cell and how it operates. But for whatever reason we still test students on memory recall of that information. Instead of asking what is the purpose of the mitochondria? we should be asking how does the cell metabolize and why are each constituents needed? This demonstrates a deep and comprehensive knowledge of the inner working of the cell. Not only that, comprehension of this aspect of the cell can lead to developments in areas such as genetic engineering and synthetic biology.
An Anecdote: One of the classes that has really benefited from becoming open-book was my cell biology for biomedical engineers course (Let’s call it BME-200). Historically, the course was taught as a memorization galore, where the quizzes were pretty much trivia games on whether or not you knew the textbook by heart. However, for online school they shifted towards explanatory short response questions. Which I loved. One of the professors even came up with a fake protein, called bme200-gtpase (an enzyme based on the course code) where he asked how we would characterize the protein! It was amazing, I felt as though I had a genuine understanding of the material and felt confident enough that I could apply this knowledge in a laboratory. That’s how I want to feel for every course, like I have true mastery over the course content.
A Case Study: History
Go from when were the 13 colonies established? to How was the development of the 13 colonies’s mercantile economic policies a precursor to modern day Capitalism? I think that’s a more useful question because it digs at a deeper question that what’s presented at the surface.
I think that the Liberal Arts, specifically the humanities, are some of the most underrepresented and unappreciated fields in academia. A common criticism that I hear from them is that the degree in this field is useless. Which I think is false, the humanities teaches a lot of valuable skills that are extremely relevant and beneficial today. Whenever people question the utility of History specifically, I always like to think back to the old aphorism, “Those who forget history, are doomed to repeat it”. Understanding the historical development of nations, governments, and even ancient societies can give an insight for current day predicaments. For example, instead of knowing what were the 13 colonies or even when they were established, I think students should be asked How was the development of mercantile trade a precursor to Capitalism. Once they understand that, you can begin to leverage this information for other important applications (like business and civil duties). But this doesn’t have to be strictly business related, it can be relevant to any area since having a comprehensive understanding of the historical precedents puts you in an excellent position to build off of it. That’s why I think science teachers start off by teaching Newtonian mechanics and show the derivations of equations, to understand how the physics worked historically so we can better understand how to expand on it. Now, I said I was going to focus on the humanities but I’m a STEM student so I always divert back to science. In that spirit, I am going to shout out this video since I think it does a wonderful job in illustrating this idea.
Science and History are especially linked in a way that is often underappreciated. We should always try and stand on the shoulders of giants.
Conclusion
The advent of COVID-19 and the sudden rise of admissions to “Zoom University” has presented a great challenge for teachers, how do we stop cheating if we can’t directly invigilate? Not only that, it has scrutinized the mode of assessment delivery and even questioned whether or not the assessments themselves were truly reflective of student comprehension. Pedagogy is an art, something that is constantly changing and adapting to today’s problems and unique circumstances. I think that Universities and schools have done an adequate job in traversing the challenge that was the COVID-19 pandemic. However, there is still a lot that is left to be desired.
I want to end this article by talking about one of my favorite childhood films, “Taare Zameen Par.” The film is a heart warming story about a young boy named Ishaan who struggles with academics and is labeled as a problem child. However, through the help of the new art teacher, he discovers that he has a unique talent for painting and starts to flourish in a way that he never had before. To me this film isn’t about how one teacher can change the life of a student, but rather, how teachers need to change the way they approach teaching to truly help a student succeed. The film highlights the importance of recognizing and understanding the strengths and weaknesses of traditional education, and how solely traditional forms of assessment can often be limiting. “Taare Zameen Par” showcases this transformational impact that a different teaching style can have on a student, and how that teaching style should change and adapt to circumstances. “Taare Zameen Par”, to me, serves as a reminder that education is not just about academics, but also about unlocking potential and finding purpose, and that this can only be achieved through a more realistic and practical approach to teaching.
