R exam postprocessing
Following my three-fold R exam of last month, I had a depressing afternoon meeting (with other faculty members) some students who had submitted R codes that were suspiciously close to other submitted R codes… In other words, it looked very likely they had cheated. (A long-term issue with my R course, alas!) During this meeting, they actually admitted either to directly copying on their neighbour’s screen, due to the limited number of terminals that forces students to be too close to one another, or to looking at (and copying) another student’s R code file from an earlier exam. I used different exams but with enough of the same spirit that some of the R code could be recycled.) Besides the pain of having to turn to disciplinary action at a level where students should see the point of getting real skills towards an incoming hiring, the depressing consequence of this state of affairs is that next we will have to move to a higher level of “security” when running the R exam, which most likely means we will turn back to a pencil-and-paper exam…. A paradoxical situation when teaching a computer programming course! But unless some unlikely sponsor delivers a computer room able to handle 180 students all at once, I do not see any other solution. Suggestions?!
Xi'an's Og 
March 28, 2011 at 12:21 am
[…] what seems like an endless cuRse, I found this week I had to re-grade a dozen R exams a TA’s did not grade properly! The […]
March 9, 2010 at 5:09 pm
It seems to me that if this is a higher-level course, then you have too many students. If the university is not willing to give you reasonably-sized classes, and the university wants to prevent cheating, then the university needs to provide substantial disincentives for cheating, like automatic course failure, plus loss of credits for the course prerequisites. Setting a student back a year or more should prevent most cheating.
Since these are all solutions that are outside of your sphere of authority, you might instead question whether or not this sort of “cheating” is really cheating. Why do you need students to complete these exams completely independently? In their professional life, there will be few negative consequences to either reusing other’s code or co-development of solutions, and there will generally be substantial benefits (e.g. reducing time-to-completion, which in turn benefits companies via improvements in time-to-market). Code re-use and “cooperation” should, perhaps, be encouraged, with novel solutions that demonstrate independent thought and creativity receiving additional points and thereby shifting the curve; copiers won’t be “A” students. Of course, you have to balance this against the need to create intelligible, maintainable code (“independent” and “creativity” can all-to-often == “cowboy crap”).
It’s a tough problem.
March 10, 2010 at 7:40 am
Thanks for the comments, Tom. I have experimented with group projects and oral defence in the past, which was more satisfactory, until we found that some students were purchasing the complete solution from computer science students! The oral defence being necessarily short because of the large size of the class, it was then difficult to spot buyers among creators. And this was unfair to other students since the purchased codes were usually of a very high quality. As you say, there is a problem with the size of the course, otherwise we could have a better monitoring of the students and solve the issue by getting to know them better….
February 21, 2010 at 5:14 pm
Dare I suggest it, Screen Visors?
Though much more common in the heyday of CRTs attached to public-use monitors in libraries and internet cafes, if the trouble is simply one student looking at another’s code, make it impossible to see without making a huge spectacle. Or, if the university lacks the budget for visors, try taping manilla folders to the vertical edges of the monitors.
February 21, 2010 at 5:44 pm
Thanks, that would solve low-key copying. Students exchanging files via file transfers are another issue, but they can be handled by using one-event-only accounts… Xi’an
February 20, 2010 at 2:51 pm
I think randomizing the order of the questions is the best approach if you want to give ‘the same’ test to all those students. The other option is to build a bigger corpus of questions, perhaps in groups of difficulty. Select a certain number from each difficulty level. Most modern test databases can help select questions in this manner.
Pencil and Paper tests have never been satisfactory for programming exams, as programming inherently involves some trial and error (or iterative improvement of working code to get optimal or elegant code), and that is part of the learning process.
February 20, 2010 at 12:40 am
You can create parallel forms of questionnaires, but at cost of putting more effort in constructing those questionnaires. With this approach, you’ll bypass copying (or at least make it more difficult), and still manage to do it “the modern way”.
To be more precise, you can create 4 versions of tests, (almost) equally difficult, and then distribute those in a such way that you should eliminate copying (“neighbour” students must have different questions). It requires more effort, but once you get into it, you’ll acquire some proficiency in constructing parallel forms of tests. =)
February 20, 2010 at 12:38 am
Two simple suggestions:
1. Randomise the order that questions appear in on WebCT. That should limit ‘over the shoulder’ copying.
2. Can you use set.seed(student_id) anywhere in the exam, so that all numerical answers will be different?
February 19, 2010 at 11:38 pm
One possible way would be to give three similar tests, so that no would would be sitting directly next to someone taking the same exam. Even mixed up the ordering of questions would make over-the-shoulder cheating harder.
Though, in a perfect world it would be better to assign a project that would test their knowledge of how to solve complex statistical problems in R, so cheating would be impossible. But, I know that that’s not always possible in larger classes.
February 20, 2010 at 6:48 am
Thanks to all for your comments and suggestions! When I started teaching this course, I used to give projects to pairs of students with a written part and a defence of 15mn. After a few years of this, we diwscovered that some students were using others’ codes, sometimes even paying for it… So we moved to an exam, last year on WebCT, this year on their own machine because WebCT is not well-suited for a mathematically oriented text with the inclusion of R code. (WebCT was a disappointment as an exam interface because having a web page to run the exam clearly is the solution against cheating: the students cannot transfer either the text or the solution to other students.) Creating three different tests for each session would require writing nine tests in total, thus a huge time investment, The problem with this class is that it is very large with 150 to 180 students, so working on projects with small groups and close control is impossible!, and even handling a single exam session is an issue because of the lack of machines. This year we tried to let students use their own laptop but only a dozen agreed…
February 19, 2010 at 10:04 pm
I had a couple of R questions on my pencil and paper final exam. So it is possible.
February 19, 2010 at 10:13 pm
Yes, I did run a pencil and paper backup exam last September. But this still seems soooo counterproductive to me: they will get a bad grade because they won’t be able to compile their R program and won’t then spot the silly typo on the 234th line of their program. This does not test them on their ability of being good R programmers or good modellers in the end… All this because of a few lazy guys (and gals)…