In a discussion of college education — I believe there should be more allowance for human diversity — sparked by this post, Alex Tabarrok told the following story:
Tyler [Cowen] once walked into class the day of the final exam and said, “Here is the exam. Write your own questions. Write your own answers. Harder questions and better answers get more points.” Then he walked out. The funniest thing was when a student came in late and I had to explain to him what the exam was and he didn’t believe me!
I was impressed. This approach, unlike most exams but like actual economies, rewards rather than punishes specialization. I asked Tyler what happened. He replied:
I would say that the variance of the test scores probably increased!
I don’t recall if I ever did that again for a whole exam but most of my exams do that for at least one question. It’s the question where you learn the most about the student.
it’s a good idea. my only recommendation would be to do this for an exam near the beginning of the class, just so you can act on the information you get while class is going on.
This is awesome. It’s also awesome when bloggers one has independently elected to follow post about one another.
Your link goes to a post about Valve Software, not a final exam.
Seth: The post about Valve Software led to the discussion.
[…] 2. Tyler Cowen’s unusual final exam. […]
One principle that I learned over my time doing consumer insights work is that we always tend to underestimate population variance – it is a LOT larger than we measure – Tyler’s class is one more data point in support of that theory…
this is the best idea i’ve heard of in education. i totally agree with cowen that it might not be feasible for an entire exam, but this is great.
I sat an International Political Economy exam (set by the excellent Alex Coram) in which the final question was “Write a question you wish had been in this exam, and answer it”. As I recall some students didn’t appreciate that opportunity as much as I did.
I think the approach is useful in that it rewards understanding rather than regurgitation, and that it contains a meta-question that is something like: which are the important topics in this course?
This is a good idea except that the question should be more specific and random if this tactic is used repeatedly.
e.g. “Write your own question and answer it using cost-benefit analysis” “Write your own question pertaining to business cycles and answer it”
This way students won’t be tempted to spend large amounts of time designing their own exams beforehand.
This is a similar approach to something I frequently do when interviewing candidates for jobs. The last question I’ll usually ask each one is, “Is there anything else you wish I’d asked you?”
Just personally, if I were given an exam like that, I would want a list of keywords or something for guidance. I can imagine some students panicking and completely blanking on what topics the class covered.
I was an Economics TA for a few years. The last question on my (ten-question) final exams was always “List and discuss four things you have learned in this class that were not asked about on this exam.” I generally got positive responses to that; people were eager to show off things that they had studied.
One result of this strategy is that every student will be correct in their prediction of what will be on the exam. Regardless of which material is actually most important, if a student believes X, Y, and Z will be on the exam, she will study X, Y, and Z. Faced with an open-ended exam, she will write on what she has studied, namely X, Y, and Z.
Or, in principle, another thing that might happen is that a student might write and answer a bunch of suitably difficult questions all rooted in materials from the last half of the course. That would lead to a high grade, despite ignoring (and perhaps being ignorant of) half the material. So, yes, additional constraints would be wise.
my hypothesis: he didn’t have an exam ready to give
In the 1960′s a Harvard chemistry professor posed a question on a chemistry final examination along these lines:
10% Extra Credit. Write a question to be used as an extra credit question on a final examination in chemistry. The ideal extra credit question should be worth about 10% of the grade on the examination as a whole, and test facility with the material covered during the course.
My friend Tom Hervey received full credit with the following answer:
10% Extra Credit. Write a question to be used as an extra credit question on a final examination in chemistry. The ideal extra credit question should be worth about 10% of the grade on the examination as a whole, and test facility with the material covered during the course.
I like the idea – how will the answers/questions be objectively assessed?
I once taught an upper division course. At the beginning of the course I handed out 100 questions. I told the students we will cover the answers to all 100 questions in the course. Further, each student will have 10 questions randomly chosen from the list of 100 questions on his Final Exam. I did not grade on a curve and I told them they could all get A’s or everyone could Fail. Surprisingly, the grade distribution was not different from the other courses.
tt – you evidently have no idea how fast Tyler reads and can generate content.
What is “Hard” ? Aren’t such things relative? So, evaluation would be based on the difficulty as perceived by the student or the teacher?
Seth, it amazes me that on this great blog full of great ideas you continually venerate a great big zero like Tyler Cowen. Thorstein Veblen, Jane Jacobs, Renata Adler, and … Cowen? One of these things is not like the others…
Seth: What’s wrong with this post?
A few years ago, i was all over the UK papers (some positive, most negative) and had a 2 page spread in THE after doing this; in a more structured way though. Look up “lecturer lets students set own exam” on Google for the media coverage. It worked well as a pedagogic exercise/experiment in assessment for learning. As a PR disaster for my institution, it wasnt quite so good; the first act of the newly appointed VC was to make a statement supporting me on academic grounds, which I’m sure he wasn’t pleased about. I even had a poison pen letter from a Daily Mail reader, and lost out at a subsequent job interview when it was raised
Oh well…
Seth: for example https://www.dailymail.co.uk/news/article-393053/Write-exam-lecturer-tells-students.html
I think it’s a very good idea. You completely eliminate the possibility of cheating and get to evaluate one’s understanding of the class. The smartest students would have evaluated the instructors positions on subjects and formed their exam based on what the professor believed.