Open-Internet Tests

Back in December 2010  I was was issued a challenge: "If all your tests were open-internet tests, how would you have to write them differently?"  This question has been vexing me ever since that time, and when I ask it to my colleagues I am often greeted with blank stares.

Jump forward to July 2011.  While at Alan November's conference in Boston, MA I had the opportunity to hear Stephen Wolfram give a presentation on http://wolframalpha.com in which he demonstrated how most basic recall and computational questions could be answered in seconds using the site.  At that point I knew I had to open up my Chemistry exams to the internet.

August 2011 rolled around and on the first day of school I explained to my students that all my tests would be open-internet tests.  I did not change them in any way from the previous year, I just wanted to find out how students would do on the tests compared to previous years.

It is now the last week of the semester and I have made a few observations.  I have not yet compared any test scores to previous years, but plan to do so once all the final exams have been taken and the grades are in.  However, I have noticed a few very interesting trends.  Four categories of students seem to have emerged.

1. The Purists:

2. The Opportunists:

3. The Google Challenged:

4. The Disinterested:

The Purists

These are the students who neither want nor need the additional resources available on the internet to succeed in my class.  They choose to learn how to solve the problems and understand the concepts in the same manner that students in previous years have.  I teach it, they internalize it, and can use it without other resources.  Occasionally these students will make careless errors on exams, and when asked why they didn't use the internet, the typically respond with "I don't want to, I want to be able to do it myself."

One of the objections I hear from my colleagues when I tell them I have open-internet tests is that I am helping to create a generation of people who are reliant on the internet, and soon we will have no one who can perform basic math because we will all be dependent upon calculators the internet.  I am happy to report that regardless of the availability of "cheats," some students will still want to dig in and learn all there is to learn on their own and will desire to be able to do it themselves.

The Opportunists

These students are the ones who couldn't thank me enough for opening up the internet during exams.  This group is composed of students who struggle with recall, but can perform quite well with a simple list of equations or notes to jog their memories.  Students from this group generally have found one or two resources (i.e. wolframalpha) that they have learned to use effectively to help them with the parts of the problems that require some of the lower end of Bloom's taxonomy so they can concentrate on the upper end.  I am confident that these students have benefitted most from the open-internet tests.

The Google Challenged

These students try to use the internet to help them on their tests, but simply are horrible at searching. Often times they will type in my test question in its entirety into Google and expect it to give them the answer.  They want to use the resources available, but do not know how.  For these students, I plan to spend additional time teaching them the skill of developing a strategic search strategy, generating an effective search term, searching the internet, filtering the results, and reporting those results accurately.  I find that this group is made up of the same students who work hard, but continue to struggle to put all the pieces together in other classes I have taught.  My class seems to lack the necessary relevance to help this group connect all the dots.

The Disinterested

This is the most intriguing group of students.  I call this group The Disinterested because they seem to lack the interest in the class/content/school/whatever to even bother to use the resources available to them. I have noticed that when reviewing a test with a student from this group that very simple questions with very simple answers that could be easily obtained from a very basic Google search have been missed.  When I inquire if they used the internet as a resource, the typical response is "no, I didn't feel like looking for the answer."  Clearly my exam does not accurately reflect what they can or cannot do, it simply reflects what they did or did not do.  Big difference, with loads of implications to the validity of testing in general, but I'll save that for a different post.

The letter grade breakdown:

The Purists have As

The Opportunists have As or Bs

The Google Challenged have Cs and Ds

The Disinterested have Ds or Fs

To be honest, these grades do not surprise me in the least bit.  I'm pretty sure that when I look at the grades compared to last year that I will have a few more As and Bs than before because The Opportunists have capitalized on the available resource and have improved because of it.  Everyone else...business as usual.

The Wild Cards

There is one additional group of students not addressed above.  They are the students who opted out of my tests and chose to complete an alternate assignment that demonstrated their understanding of the objective in the unit instead.  I'm still working out some of the details of how I assess these students.  Currently, I check to make sure their project or product has addressed all the objectives and demonstrates their understanding, but it's still pretty subjective.  However, most students who have gone with this option have blown me away with the creativity and quality of their work.  I will also admit that some of my students are on to the fact that things are a little squishy with these assessments and are choosing this option as an easy way out. 

Conclusions and reflections:

1. By using the same exams as in the past and opening them up to the internet, a small portion of my students have been able to improve their grades.

2. I really need to change the way I assess my students.  The Purists, The Google Challenged, and The Disinterested have not benefitted from open-internet tests.  I need a better way to assess the understanding of the Google Challenged and the Disinterested, and I need to be able to explore the upper limits of The Purists.

3. My students continue to treat tests as a means to acquire the sufficient number of points to obtain the letter grade that will keep their parents off of their backs.  They do not treat them as the diagnostic tool needed to help me determine what a student knows and can do.

4. I'm tempted to dump my summative unit tests entirely for the second semester and require that all students create a project or product that demonstrates their understanding of the objectives in the course.  I also hope to incorporate more project driven units that are less content focused, and more context driven.

5. I need to amp up the quality control on the alternate assessment, especially if I only accept this form of assessment in the second semester.

I would appreciate any comments or feedback.