Open-Internet Tests

Back in December 2010  I was was issued a challenge: "If all your tests were open-internet tests, how would you have to write them differently?"  This question has been vexing me ever since that time, and when I ask it to my colleagues I am often greeted with blank stares.

Jump forward to July 2011.  While at Alan November's conference in Boston, MA I had the opportunity to hear Stephen Wolfram give a presentation on http://wolframalpha.com in which he demonstrated how most basic recall and computational questions could be answered in seconds using the site.  At that point I knew I had to open up my Chemistry exams to the internet.

August 2011 rolled around and on the first day of school I explained to my students that all my tests would be open-internet tests.  I did not change them in any way from the previous year, I just wanted to find out how students would do on the tests compared to previous years.

It is now the last week of the semester and I have made a few observations.  I have not yet compared any test scores to previous years, but plan to do so once all the final exams have been taken and the grades are in.  However, I have noticed a few very interesting trends.  Four categories of students seem to have emerged.

1. The Purists:

2. The Opportunists:

3. The Google Challenged:

4. The Disinterested:

The Purists

These are the students who neither want nor need the additional resources available on the internet to succeed in my class.  They choose to learn how to solve the problems and understand the concepts in the same manner that students in previous years have.  I teach it, they internalize it, and can use it without other resources.  Occasionally these students will make careless errors on exams, and when asked why they didn't use the internet, the typically respond with "I don't want to, I want to be able to do it myself."

One of the objections I hear from my colleagues when I tell them I have open-internet tests is that I am helping to create a generation of people who are reliant on the internet, and soon we will have no one who can perform basic math because we will all be dependent upon calculators the internet.  I am happy to report that regardless of the availability of "cheats," some students will still want to dig in and learn all there is to learn on their own and will desire to be able to do it themselves.

The Opportunists

These students are the ones who couldn't thank me enough for opening up the internet during exams.  This group is composed of students who struggle with recall, but can perform quite well with a simple list of equations or notes to jog their memories.  Students from this group generally have found one or two resources (i.e. wolframalpha) that they have learned to use effectively to help them with the parts of the problems that require some of the lower end of Bloom's taxonomy so they can concentrate on the upper end.  I am confident that these students have benefitted most from the open-internet tests.

The Google Challenged

These students try to use the internet to help them on their tests, but simply are horrible at searching. Often times they will type in my test question in its entirety into Google and expect it to give them the answer.  They want to use the resources available, but do not know how.  For these students, I plan to spend additional time teaching them the skill of developing a strategic search strategy, generating an effective search term, searching the internet, filtering the results, and reporting those results accurately.  I find that this group is made up of the same students who work hard, but continue to struggle to put all the pieces together in other classes I have taught.  My class seems to lack the necessary relevance to help this group connect all the dots.

The Disinterested

This is the most intriguing group of students.  I call this group The Disinterested because they seem to lack the interest in the class/content/school/whatever to even bother to use the resources available to them. I have noticed that when reviewing a test with a student from this group that very simple questions with very simple answers that could be easily obtained from a very basic Google search have been missed.  When I inquire if they used the internet as a resource, the typical response is "no, I didn't feel like looking for the answer."  Clearly my exam does not accurately reflect what they can or cannot do, it simply reflects what they did or did not do.  Big difference, with loads of implications to the validity of testing in general, but I'll save that for a different post.

The letter grade breakdown:

The Purists have As

The Opportunists have As or Bs

The Google Challenged have Cs and Ds

The Disinterested have Ds or Fs

To be honest, these grades do not surprise me in the least bit.  I'm pretty sure that when I look at the grades compared to last year that I will have a few more As and Bs than before because The Opportunists have capitalized on the available resource and have improved because of it.  Everyone else...business as usual.

The Wild Cards

There is one additional group of students not addressed above.  They are the students who opted out of my tests and chose to complete an alternate assignment that demonstrated their understanding of the objective in the unit instead.  I'm still working out some of the details of how I assess these students.  Currently, I check to make sure their project or product has addressed all the objectives and demonstrates their understanding, but it's still pretty subjective.  However, most students who have gone with this option have blown me away with the creativity and quality of their work.  I will also admit that some of my students are on to the fact that things are a little squishy with these assessments and are choosing this option as an easy way out. 

Conclusions and reflections:

1. By using the same exams as in the past and opening them up to the internet, a small portion of my students have been able to improve their grades.

2. I really need to change the way I assess my students.  The Purists, The Google Challenged, and The Disinterested have not benefitted from open-internet tests.  I need a better way to assess the understanding of the Google Challenged and the Disinterested, and I need to be able to explore the upper limits of The Purists.

3. My students continue to treat tests as a means to acquire the sufficient number of points to obtain the letter grade that will keep their parents off of their backs.  They do not treat them as the diagnostic tool needed to help me determine what a student knows and can do.

4. I'm tempted to dump my summative unit tests entirely for the second semester and require that all students create a project or product that demonstrates their understanding of the objectives in the course.  I also hope to incorporate more project driven units that are less content focused, and more context driven.

5. I need to amp up the quality control on the alternate assessment, especially if I only accept this form of assessment in the second semester.

I would appreciate any comments or feedback.

Things I Learned In Norway

I recently traveled to Norway to present at an education conference in Sandvika, just outside Oslo.  I have traveled internationally quite a bit in my 34 years on this planet, but I had not been outside North America since 2004.  My reflections are as follows:

1. If you don't like American football (I don't) and do like real football (I do), you will be mistaken for a Canadian (not that there's anything wrong with that).

2. The US banks need to get with the times and issue Chip and Pin credit and debit cards.  When the waiter at the restaurant calls swiping your card "running it the old fashioned way," and your colleague cannot pay at the gas pump, something should probably give.

3. I am thankful for and humbled by how much of the world speaks English, and I wish I knew more languages.  American schools need to start teaching languages earlier.

4. I submitted a bill to the organization I was working with for some travel reimbursement and was notified that Norwegian banks no longer issue checks and the money would have to be wired electronically.  No big deal, right?  Come to find out, the small local credit union I use does not have an international SWIFT number necessary to handle the transaction.  I like this particular local credit union, but I guess shopping local can have some limitations.

5. Educators in Norway are highly professional, take pride in their career choice, and are compensated adequately.  I noticed the same when I was in Victoria, BC last year.

6. Trips to interesting places are much better when my wife comes along.  I'm glad she got to join me on this trip.

7. Fish is best served raw, cured, at breakfast, and often.

8. IKEA in Norway looks, feels and smells like IKEA in the US.

Pedagogy Must Drive Technology

It seems that the Ed Tech mantra of late is that "Pedagogy Must Drive Technology."  I wholeheartedly agree with this statement, and would like to highlight a few examples of where I have seen individuals apply this statement under the umbrella of The Flipped Classroom.  But more importantly, I wanted to show that the pedagogy behind the "traditional" Flipped Class model (homework becomes classwork, classwork becomes homework) is not novel or new, and also show that the Flipped Class is not a pedagogy or methodology in and of itself, it is a tool in the toolbox of educators.

As a side note, I have spent time getting to know each of these educators personally, and I am honored to call each of them a colleague.

This post was inspired by a recent Google+ conversation I saw between Ron Houtman and Greg Green.

As Ron and Greg point out above, giving students something to do prior to coming to class is not new or novel. For centuries, students read, researched, studied at home and came to class to discuss, question and explore.  Under the "traditional" Flipped Class model, students do just that, but have a media rich video as an resource for instruction in addition to a textbook, the internet, and other media.  Expecting students to come to class prepared to discuss and work is NOT NEW.  Utilizing a video as one of the tools to do so IS NEW.

Now, to highlight some specific ways that educators are utilizing screencasts as an instructional tool to meet an identified student need.

1. Ron Houtman @ronhoutman: Teacher of Teachers in Michigan.  Ron has been utilizing screencasting technology since around 2000.  As far as I know, he was one of the first educators to leverage screencasts as an instructional tool.  As you can see from the conversation above, he began screencasting to help his students who missed class stay caught up.  Ron didn't screencast to use a novel new technology, he did so to meet an educational need of his students.

2. April Gudenrath @agudteach: Teacher of Literature and English in Colorado Springs, CO.  April found that giving her students meaningful feedback on papers was difficult with only a pen and paper.  She began to screencast her grading sessions so students could hear her voice and follow her thought process as she annotated the student paper.  April didn't begin screencasting to "go paperless," she did so to meet an educational need of her students.

3. Greg Green @flippedschool: Principal in Michigan.  Greg is known for using the Flipped Classroom model of pre-recording lessons to free up time in class in his entire high school.  He found that too many students were disengaged and failing, and that most students did not have the support network at home necessary to complete assignments at home.  So, he decided to bridge this gap by making all work done in class where an expert was available to assist the students.  In order to avoid creating a digital divide by delivering instruction at home he has made sure that all students have adequate technological access to the institutional screencasts.  Greg didn't screencast to try to create a high-tech high school, he did so to meet the educational needs of his students.

4. Brian Bennett @bennettscience: Science Teacher in Indiana.  Brian taught in South Korea and recently moved back to the US and teaches in Indiana.  He was using a Flipped Classroom model in Korea with great success, but noticed that his students in the US were not as successful under the same model.  So, Brian changed the role of the screencasts in his class.  Instead of using them to front-load instruction, he used them as remediation and re-teaching tools with greater success.  I regularly read his blog and follow his thoughts on Twitter and have noticed that Brian continually tries new ideas, reflects on his practices, and strives to daily meet the needs of his students.  Brian did not create screencasts for his students and blindly continue to use them when they weren't effective instructional tools.  He recognized the limitations of the screencasts in his new educational setting and modified his practice accordingly to meet the educational needs of his students.

5. Kevin Byers @kevinbyers: From his Twitter profile: "I used to teach science, technology, AVID, and then math. Now I am working to bring anywhere, anytime learning to our district." Kevin works in a school district in the Denver, CO area in which the entire district has adopted a Standards Based Grading system in which students learn at a level that is appropriate for that individual.  All classes are heterogeneous with students at different levels, and each student is likely at a different level in each subject.  This district has decided that screencasts will be an effective tool to deliver asynchronous instruction to their students.  Kevin helps oversee and coordinate the screencasting project.  Kevin and his district did not decide to use screencasts as a novel way to deliver content, he/they saw a need and leveraged the appropriate technology to meet the needs of students.

I wanted to highlight these individuals to honor the amazing work they do each day to meet the needs of students and to demonstrate once again that there is no such thing as THE Flipped Classroom.  If there was, all these educators would be using screencasting technology the same way to accomplish the same goal.  Instead, each of them has identified a need and has leveraged a technological tool to meet that need.

I welcome other stories in the comments.

There Is No Such Thing as THE Flipped Class

The term "Flipped Classroom" is being thrown around a lot lately in both positive and negative light. I think the term is a bit ambiguous and does not fully do justice to all that is being done under the guise of the Flipped Classroom. My colleague, Jon Bergmann, and I have a book coming out soon that I hope brings clarity to what most of us mean by "The Flipped Classroom." In the mean time, I hope to shed some light on some of the confusion, critique, and hype.

1. What's in a name?
There is no such thing as THE Flipped Classroom. The Flip has many faces and the word Flip has certain connotations that do not do justice to the amazing educational uses of screencasting and other video production technology.  When Jon and I began promoting the idea of using screencasting as an educational tool in 2006-2007, we struggled to know what to call the model.  At that time we settled on the name (and website URL) Educational Vodcasting.  While this name encompassed our content delivery model, it also left itself open to various applications of screencasting in education that were not restricted to delivering direct instruction through a video to be watched at home.  We quickly found that the terms "podcasting" and "vodcasting" scared away many teachers and parents.

A few years passed, our model morphed from content delivery via video, to a flex-paced mastery system and the name shifted to Reverse Instruction.  Others began utilizing screencasting technology to create short how-to tutorials for students and colleagues, others were using the technology to provides students with feedback about written essays, still others were using screencasting as a tool for remediation, re-teaching, and filling in gaps in understanding.  These different applications of screencasting, are obviously not the same, and although they use similar technology to create the videos, the classroom applications are as diverse as the teachers who use them.

Another year passed and we began to include elements of UDL and inquiry in our model.  Others, like Ramsey Musallam, listened to the critiques of educators like Frank Noschese and completely integrated inquiry learning with the instructional videos with his Explore-Flip-Apply model.

And then the "Flip" word was used.  Late in 2010, Dan Pink wrote an article for the Telegraph in which he mentioned educator and ed-tech guru Karl Fisch.  Karl had recently returned to the classroom and was using screencasting technology to deliver instruction to his students outside of class.  Pink referred to this as the "Fisch Flip."  Karl kindly credited Jon and I with inspiring him to adopt this model, and we immediately became affiliated with the "Flipped Classroom," and the name seems to have stuck.  We were in the process of writing our book at this time, and decided to call it "The Flipped Classroom," and in doing so we also stuck ourselves to the name.  We submitted our manuscript to our publisher in early Feb. 2011, and shortly thereafter Sal Khan gave a Ted Talk in which he referred to "Flipping the Class"

Here is the problem with the term "Flipped Class:" it implies version one of our screencasting model: that which used to be done in class is now done at home, and that which used to be done at home, is now done in class.  In a nutshell, that IS "The Flipped Classroom," but it does not end there, which is why the term "The Flipped Classroom" does not do justice to the many models being used.  "The Flipped Classroom" evokes images of students glued to their computers, frantically taking notes at home, coming to class, banging out worksheets, taking tests online until they "pass" an objective, unlocking the next task, lather, rinse, repeat.  And I will admit that I had my students doing exactly that for the first year I rolled out the Flipped-Mastery model (2008-2009).  But a lot of time has passed, and I have learned from my mistakes, I have learned from my Twitter PLN, and Flipped-Mastery has undergone many iterations since then.

Here is a summary of the 6 models I have used over the past 6 years:

2006-2007 Live Recording

2007-2008 Flipped

2008-2009 Flipped-Mastery

2009-2010 Flipped-Mastery/Inquiry/SBG

2010-2011 Flipped-Mastery/Inquiry/SBG/UDL

2011-2012 Flipped-Mastery/Inquiry/SBG/UDL/WolframAlpha (open-internet tests)/PBL

You can watch my presentation at the American Chemical Society this past November to hear a brief summary of each of these models here http://youtu.be/HLLciZdUpDc , and if you have watched this, you know that it would be difficult to call my classroom "THE Flipped Class."  Yes, I use video to deliver instruction; no, students are not required to watch MY videos; yes, students can learn the objectives of the class in any way they want; no, all students do not have to take MY online exam; yes, students can demonstrate their understanding to me in alternate ways; yes, I believe in inquiry; yes, students are learning; yes, they love the flexibility of the class; and yes, it works...but no, it is not perfect and I can always improve.

2. Sal + TED does not equal Flip

Just because Sal Khan used the term "Flip" in his Ted talk does not mean that the Khan Academy epitomizes the Flipped Classroom. I can't say this enough: Kahn Academy represents A form of the Flipped Classroom, it is not THE flagship of The Flipped Class concept.  Hundreds of teachers across the US and Canada were using screencasting and mastery learning years before Khan's Ted Talk in Feb 2011.  He has received a lot of attention, he has received a lot of money, but he is A voice of the Flipped Classroom, not THE voice of the Flipped Classroom.


3. The Flip is in flux
It would be foolish for any educator to adopt a model of instruction and never evaluate the efficacy of the model. This goes for Flipped Class, Inquiry, lecturing, Unschooling, or whatever educational model you use.  I have been a teacher for 12 years, and I have modified my instructional practices every year based on my own reflection, feedback from students and emerging educational practices.  Anyone who blindly adopts "The Flipped Classroom" (or inquiry, or lecturing, or unschooling, or whatever) model and never modifies it to meet the needs of his or her students will blindly lead his or her students into educational ruin.

This is why I have adapted my Flipped model every year. My Flip is in flux, which is yet another testament to the fact that there is no such thing as THE Flipped Classroom.  Brian Bennett said it best when he said "The Flipped Class is not a methodology, it is ideology."  Now, please allow me to paraphrase that statement sans buzz words: "using screencasting technology is not a one-size fits-all methodology to be rolled out on a large scale because it would be foolish to use this tool when it is not appropriate to do so; it is tool in the toolbox of education that prevents a teacher from wasting class time lecturing, (but it allows the teacher to maintain the use of appropriate direct instruction) and spends class time meeting the individual needs of students."  What the class time looks like is a wildcard dependent upon the teacher, the school, the school culture, current educational research, etc.

4. Be Specific
 When promoting or critiquing the "Flipped Classroom" please be specific about what permutation you are promoting or critiquing.

Proponents:
Do you think Sal Khan is the greatest educator of all time? Please sing his praises, but do not confuse his model with all who operate under the Flipped Class moniker, and do not assume that all who Flip do so in the same way that Los Altos High School has.  Have you created instructional videos you are proud of? By all means, please share your videos with others, but share them as tools to accomplish a particular task.  Did your department or school decide that all direct instruction will be delivered through teacher-created screencasts? Please share your exciting story, and be specific about the transformations your classroom has undergone.

Critics:
Are you being critical of the use of ANY form of direct instruction? Then please be critical of the use of direct instruction in the form of a video. Do you take issue with a teacher deciding what a student should learn? Then please critique those who establish learning objectives for their students instead of letting the students decide what to learn.  Are you opposed to a mastery model that does not allow a student to progress until they have demonstrated understanding on a particular assessment? Then please deliver your criticism to those who assess students in this way.  A blanket critique of the Flipped Classroom does not address the nuances that are present in the various applications of the Flip.

The moral of the story:
When you read anything about The Flipped Classrom mentally substitute "a class that uses screencasts as an instructional tool" for "the Flipped Classroom" and all will be well.  Don't make assumptions, don't make blanket statements, disagree with specific points, make specific assertions, and do what's best for kids.

I just gotta write something

I recently moved within a mile of the school where I work and have really enjoyed walking to work most days. This has been both a blessing, and a curse.  I love that I have time to wake up with a refreshing walk in the morning, but I now have roughly 30 minuets of uninterrupted thinking time every day. As a result, I have a lot on my mind, and I intend to bang much of it out on my recently resurrected blog.

 I feel like a walking paradox, I need to process and get some things worked out of my system, so expect future topics like:
1. I am an outspoken voice for the Flipped Classroom concept, but I hate the phrase "Flipped Classroom."
2. What I really think about The Khan Academy, and why I love and hate it all at once.
3. Why I teach public school, serve on the board of directors of a private school, but homeschool my own children.
4. This is my last year (at least for the next 6 years) as a classroom teacher, and why I'm taking a break.
5. Why I am both excited and frightened for education in the next 15 years.
6. Why I long for simplicity.

 Thank you for reading this rather pointless post, please provide me with feedback on what you would like to hear from me.  I'm sure I need to tweak some blog settings, so let me know.