Sunday, January 20, 2008
The birth of "A Video Series on Student Misconceptions"
The videos were a hit and as Apoorva explained Sridhar and I were working on a proposal for a project around misconceptions. So after a couple of months of planning, discussions we got the project approved.
What is the project all about?
What is the project all about?
Student Learning in the Metros study 2006 (SLIMS 2006) , jointly conducted by EI and Wipro, received an overwhelming response from educationists, experts, schools and parents. The November 27, 2006 issue of the India Today Magazine carried a cover story based on the findings of this research study.
The study suggested that students across classes answer rote-based or procedural questions relatively well but fail to answer unfamiliar question types correctly. Students also tend to perform poorly on questions that require practical competencies. The above shows that in many cases, they are not learning with understanding.
To shed further light a Video Series on Student Misconceptions was designed.
What are students interviews and why ?
Interesting patterns relates to misconceptions, where the number of students giving the most common wrong answer far exceeds the number of students giving the correct answer. In such cases, it is likely, though not strictly necessary, that the wrong answer emerges from a false preconception, or a common misconception. However data and numbers are just indicators and they tell us nothing about what goes on in student’s minds while they select the wrong options. To know and understand the various sources of these false preconceptions or wrong notions, its important that we talk to them and understand the various ideas they have about the question and topic being asked. It is only through such interactions with them that we can know the thought process that operates in their mind.
These interactions are called student interviews. Students are presented with a question, one which data indicates that students seem to have a misconceptions. Background research on this question and the concept is conducted before a systematic interview is conducted.
Student interviews were started with a view to understand how children think. But our experience has generated some interesting discussions with teachers and students . In some cases we have seen students change their thoughts and often learn in the process.
What are students interviews and why ?
Interesting patterns relates to misconceptions, where the number of students giving the most common wrong answer far exceeds the number of students giving the correct answer. In such cases, it is likely, though not strictly necessary, that the wrong answer emerges from a false preconception, or a common misconception. However data and numbers are just indicators and they tell us nothing about what goes on in student’s minds while they select the wrong options. To know and understand the various sources of these false preconceptions or wrong notions, its important that we talk to them and understand the various ideas they have about the question and topic being asked. It is only through such interactions with them that we can know the thought process that operates in their mind.
These interactions are called student interviews. Students are presented with a question, one which data indicates that students seem to have a misconceptions. Background research on this question and the concept is conducted before a systematic interview is conducted.
Student interviews were started with a view to understand how children think. But our experience has generated some interesting discussions with teachers and students . In some cases we have seen students change their thoughts and often learn in the process.
Wednesday, January 16, 2008
When Curiosity Created The Cat: The history of misconceptions at EI
It all started off with dissatisfaction.
Every year, EI would produce these wonderful ASSET papers and thousands of children all around the country would answer them. The answer sheets would come in and our systems would churn out pages and pages of reports. But what was the point? Did these reports say something useful about the children?
Nishchal and Apoorva often wondered whether these reports made any sense. Could we really say that a child was strong or weak at reasoning? They decided to do a simple test. They would sit down with the question paper and the answer sheet of one child from Ahmedabad. They would go through all the answers, and predict what the child would be like. Then, they would go and meet that child!
So, one afternoon, both of them and Mehrab, who was visiting, sat down in Sridhar's office (he wasn't around), and picked up paper 369 - the science paper of class 6 from the summer of 2005 and the answer sheet of a child. They first went through the answer sheet, and each wrote down what the child's answer means. Then they discussed each question one by one, slowly trying to build a portrait of the child in our minds. The girl whose answer sheet they had picked, hadn't done too well, but many of her wrong answers were pretty rational....which suggested that she probably had good reasoning skills. It was time to ask find out! But, the lazy bums that they were, they never got down to meeting the girl.
The Most Common Wrong Answer....Misconception!
But, this exercise sparked gave them a new insight. They realised that the really interesting information is not whether a child has answered incorrectly, but what option he or she has chosen. This was by no means a new insight. The seniors at EI knew the power of this approach even before they started ASSET. That was the whole point of designing good distractors. However, in those days the data wasn't easy to access and use, and analysis was rarely discussed.
But, for the naive young enthusiasts, this was like an exciting discovery! They immediately started looking at data with a renewed vigour. What percentage of students had answered each option? What did the different patterns mean?
One evening, they were passionately discussing a problem: out of the data of hundreds of questions, which one would be interesting? They realised that the questions to look at were those in which many students had selected one particular wrong answer. During a mid-road conversation with him riding his legendary scooter, Nishchal asked....."What would you call such questions?" and then immediately and emphatically pointed out, "Misconceptions!" Aha!
Over the next 3 days, they took this idea to the next stage, creating a lovely little excel file called "Misconceptions Dhoondega.xls" that would point out questions where a lot of students had gone for a common wrong answer. Sridhar, disappointed that we were using excel, which was prone to human error, quickly commissioned a system and gave us a neat suggestion on how to use standard deviations to find the most common wrong answer. Rahul, with hundreds of things on his plate, grudgingly gave in to our pestering and put together the misconceptions program, in all its green and red finery. Misconceptions were here to stay.
There is no cure for curiosity....
The misconceptions program raised more questions than it answered. As more and more common wrong answers came to the fore, it was often not clear why children would choose that option. Some of the cases were completely bizzare. For instance, more than 2500 9 year olds felt that hair on the soles of the Fennc fox helped it lose heat. Huh! The curiosity was killing!
As they say, the cure for boredom is curiosity but there is no cure for curiosity. The young enthusiasts decided to go and talk to children. One day, sometime in April 2006, Nishchal, Stuti and Apoorva finally got a chance to interview students at Riverside School in Ahmedabad. Not knowing what to expect, they just fished out some questions in which the school had common wrong answers, and decided to go and interview the kids. And, that day, they just happened to carry a camera with them!
This interview was an eye-opener. They had gone in with some hypotheses about why kids select certain options and while some of them were dramatically confirmed, others were completely thrown out. They found, for instance, that a key below a graph was completely missed by children. They would read the entire question and completely skip it. In another question, they came across a child who thought that clouds are alive! After probing, they realised that he thought clouds were alive because they had to 'let water go'! The answers were coming! The game was afoot!
And they believed: the power of video
The most dramatic development in the saga was yet to happen. In the 2006 EI connect, Apoorva was given 10 minutes to summarise the work on the misconceptions program that had happened 'behind the scenes'. As usual, he did not prepare for the presentation! He just took a screen capture of the misconception program, and the raw video from the Riverside interviews and decided to finish the story.
Almost all of EI was packed into one long conference room. The guests had arrived - Prof. Rama Mathew from Delhi and Kiranbhai from Rajkot. It was the first or the second session and Apoorva was hoping to get over with it without boring anyone. First, he explained the misconceptions program and what it did. A question here and there, but no real response. The audience was bored with the screen full of data. Then, he showed the graph question. Some interest. Then, he played the video. The child comes up, explains his answer. People laugh. Then, the interviewer asks the next question - "Can you please read the entire question". Child reads. Audience watches. Child skips the "key". Bang! Suddenly, everyone was excited. Sridhar started laughing, comments started flowing. VS made a point about children self-learning and the hole-in-the-wall experiment. Sales team was murmuring. People broke for coffee and the conversation continued.
It was clear that the interview video was a powerful way of bringing out what was going on in the child's mind. The child explaining his/her reasoning got across to people in a way that data never did. This power had to be channelised.
The First Films: back on the radar
Clearly, this first 'video' was an accident. It wasn't even cut into a clip. Apoorva had just played the raw footage. Accidents can get things started, but one has to persist to get anywhere. For several months, nothing happened. The momentary excitement had died down, and the idea was forgotten.
It took another child speaking his mind to kickstart the idea again. The occassion was the ASSET test in August, 2006. Nishchal and Apoorva decided to visit schools while the test happened, believing that this will give some insight into how the testing process. The visits, as expected, were insightful and one exciting moment kickstarted the interviews again. Apoorva was talking to some children from class 7, who had just finished the science test. They were discussing a question about how much time a crumpled paper would take to fall, compared to an uncrumpled one. The children had all answered correctly, that the plain paper would take longer. But, their reason was a shocker! They felt that it was because the crumpled paper weighed more!
Again, talking to children had provided an interesting insight, even indicating that one cannot assume that a correct answer is based on sound reasoning. This time, the momentum generated was going to move things forward. An opportunity emerged in a school to conduct interviews with children. Anar and Apoorva jumped at the chance, and decided to conduct full-fledged interviews and videoshoot them. These interviews were relly done behind the scenes, with only Nishchal and a few others aware of them. Amazingly, even a mail wasn't sent reporting what happened!
With Anar's expertise in videography and editing, they sat down and put together the first two 'misconception films' just in time for the fist Test Development meeting in the middle of September. The films were shown and immediately met with intense criticism! The expert view was that the interviews weren't conducted carefully, and the questions weren't right, and the interviews weren't shared! However, interviews and videos were now on people's radar, and there was no stopping. Interviews with more schools were planned and lots of experimentation happened over the next few months.
Another critical development was that the sales team got involved. There was a post-ASSET workshop scheduled at the school that the films were based on. Sudhir and Apoorva put together a 2-hour session, which involved the teacher seeing interviews of their own students and the message was driven home. Baiju, who was present, could see the power of the video in getting through the customer. Support for interviews and videos was getting bigger.
Sridhar and the scaleup: the birth of the Video Series on Student Misconceptions
However, interviews were still a niche activity that only people with a lot of interest engaged in. While there was a demand from the sales team for more and more videos, few people actually conducted interviews! At this stage, Sridhar took the bull by the horns and decided to do something that would scale up the interviews. He decided to write a proposal for a project which would involve interviews in schools all over the country and asked a leading IT company for funding. If this came through, resources would become available for systematically conducting interviews and producing films. Sridhar and Anar were working hard on the proposal. However, the key was convincing the team from the IT company, who were visiting soon.
Another interview was planned at the same school. Vishnu, Anar, Apoorva and the three guests from the IT company joined a teacher a group of 30 children for an amazing two hour experience. As Vishnu and Apoorva, with months of ASSET-linked curiosity bottled up, started interviewing the kids, one after another amazing moment unfolded. Children were thinking, talking, enjoying, and learning while we slowly got a glimpse into their minds, discovering misconceptions and hidden gems. Sadly, only one films was ever made from these interviews...but it was a runaway hit! The Perimeter Problem in which Vishnu brought together a triangular island, a tsunami and a goat to create magic.
The project came through....and the rest, as they say, is history!
Every year, EI would produce these wonderful ASSET papers and thousands of children all around the country would answer them. The answer sheets would come in and our systems would churn out pages and pages of reports. But what was the point? Did these reports say something useful about the children?
Nishchal and Apoorva often wondered whether these reports made any sense. Could we really say that a child was strong or weak at reasoning? They decided to do a simple test. They would sit down with the question paper and the answer sheet of one child from Ahmedabad. They would go through all the answers, and predict what the child would be like. Then, they would go and meet that child!
So, one afternoon, both of them and Mehrab, who was visiting, sat down in Sridhar's office (he wasn't around), and picked up paper 369 - the science paper of class 6 from the summer of 2005 and the answer sheet of a child. They first went through the answer sheet, and each wrote down what the child's answer means. Then they discussed each question one by one, slowly trying to build a portrait of the child in our minds. The girl whose answer sheet they had picked, hadn't done too well, but many of her wrong answers were pretty rational....which suggested that she probably had good reasoning skills. It was time to ask find out! But, the lazy bums that they were, they never got down to meeting the girl.
The Most Common Wrong Answer....Misconception!
But, this exercise sparked gave them a new insight. They realised that the really interesting information is not whether a child has answered incorrectly, but what option he or she has chosen. This was by no means a new insight. The seniors at EI knew the power of this approach even before they started ASSET. That was the whole point of designing good distractors. However, in those days the data wasn't easy to access and use, and analysis was rarely discussed.
But, for the naive young enthusiasts, this was like an exciting discovery! They immediately started looking at data with a renewed vigour. What percentage of students had answered each option? What did the different patterns mean?
One evening, they were passionately discussing a problem: out of the data of hundreds of questions, which one would be interesting? They realised that the questions to look at were those in which many students had selected one particular wrong answer. During a mid-road conversation with him riding his legendary scooter, Nishchal asked....."What would you call such questions?" and then immediately and emphatically pointed out, "Misconceptions!" Aha!
Over the next 3 days, they took this idea to the next stage, creating a lovely little excel file called "Misconceptions Dhoondega.xls" that would point out questions where a lot of students had gone for a common wrong answer. Sridhar, disappointed that we were using excel, which was prone to human error, quickly commissioned a system and gave us a neat suggestion on how to use standard deviations to find the most common wrong answer. Rahul, with hundreds of things on his plate, grudgingly gave in to our pestering and put together the misconceptions program, in all its green and red finery. Misconceptions were here to stay.
There is no cure for curiosity....
The misconceptions program raised more questions than it answered. As more and more common wrong answers came to the fore, it was often not clear why children would choose that option. Some of the cases were completely bizzare. For instance, more than 2500 9 year olds felt that hair on the soles of the Fennc fox helped it lose heat. Huh! The curiosity was killing!
As they say, the cure for boredom is curiosity but there is no cure for curiosity. The young enthusiasts decided to go and talk to children. One day, sometime in April 2006, Nishchal, Stuti and Apoorva finally got a chance to interview students at Riverside School in Ahmedabad. Not knowing what to expect, they just fished out some questions in which the school had common wrong answers, and decided to go and interview the kids. And, that day, they just happened to carry a camera with them!
This interview was an eye-opener. They had gone in with some hypotheses about why kids select certain options and while some of them were dramatically confirmed, others were completely thrown out. They found, for instance, that a key below a graph was completely missed by children. They would read the entire question and completely skip it. In another question, they came across a child who thought that clouds are alive! After probing, they realised that he thought clouds were alive because they had to 'let water go'! The answers were coming! The game was afoot!
And they believed: the power of video
The most dramatic development in the saga was yet to happen. In the 2006 EI connect, Apoorva was given 10 minutes to summarise the work on the misconceptions program that had happened 'behind the scenes'. As usual, he did not prepare for the presentation! He just took a screen capture of the misconception program, and the raw video from the Riverside interviews and decided to finish the story.
Almost all of EI was packed into one long conference room. The guests had arrived - Prof. Rama Mathew from Delhi and Kiranbhai from Rajkot. It was the first or the second session and Apoorva was hoping to get over with it without boring anyone. First, he explained the misconceptions program and what it did. A question here and there, but no real response. The audience was bored with the screen full of data. Then, he showed the graph question. Some interest. Then, he played the video. The child comes up, explains his answer. People laugh. Then, the interviewer asks the next question - "Can you please read the entire question". Child reads. Audience watches. Child skips the "key". Bang! Suddenly, everyone was excited. Sridhar started laughing, comments started flowing. VS made a point about children self-learning and the hole-in-the-wall experiment. Sales team was murmuring. People broke for coffee and the conversation continued.
It was clear that the interview video was a powerful way of bringing out what was going on in the child's mind. The child explaining his/her reasoning got across to people in a way that data never did. This power had to be channelised.
The First Films: back on the radar
Clearly, this first 'video' was an accident. It wasn't even cut into a clip. Apoorva had just played the raw footage. Accidents can get things started, but one has to persist to get anywhere. For several months, nothing happened. The momentary excitement had died down, and the idea was forgotten.
It took another child speaking his mind to kickstart the idea again. The occassion was the ASSET test in August, 2006. Nishchal and Apoorva decided to visit schools while the test happened, believing that this will give some insight into how the testing process. The visits, as expected, were insightful and one exciting moment kickstarted the interviews again. Apoorva was talking to some children from class 7, who had just finished the science test. They were discussing a question about how much time a crumpled paper would take to fall, compared to an uncrumpled one. The children had all answered correctly, that the plain paper would take longer. But, their reason was a shocker! They felt that it was because the crumpled paper weighed more!
Again, talking to children had provided an interesting insight, even indicating that one cannot assume that a correct answer is based on sound reasoning. This time, the momentum generated was going to move things forward. An opportunity emerged in a school to conduct interviews with children. Anar and Apoorva jumped at the chance, and decided to conduct full-fledged interviews and videoshoot them. These interviews were relly done behind the scenes, with only Nishchal and a few others aware of them. Amazingly, even a mail wasn't sent reporting what happened!
With Anar's expertise in videography and editing, they sat down and put together the first two 'misconception films' just in time for the fist Test Development meeting in the middle of September. The films were shown and immediately met with intense criticism! The expert view was that the interviews weren't conducted carefully, and the questions weren't right, and the interviews weren't shared! However, interviews and videos were now on people's radar, and there was no stopping. Interviews with more schools were planned and lots of experimentation happened over the next few months.
Another critical development was that the sales team got involved. There was a post-ASSET workshop scheduled at the school that the films were based on. Sudhir and Apoorva put together a 2-hour session, which involved the teacher seeing interviews of their own students and the message was driven home. Baiju, who was present, could see the power of the video in getting through the customer. Support for interviews and videos was getting bigger.
Sridhar and the scaleup: the birth of the Video Series on Student Misconceptions
However, interviews were still a niche activity that only people with a lot of interest engaged in. While there was a demand from the sales team for more and more videos, few people actually conducted interviews! At this stage, Sridhar took the bull by the horns and decided to do something that would scale up the interviews. He decided to write a proposal for a project which would involve interviews in schools all over the country and asked a leading IT company for funding. If this came through, resources would become available for systematically conducting interviews and producing films. Sridhar and Anar were working hard on the proposal. However, the key was convincing the team from the IT company, who were visiting soon.
Another interview was planned at the same school. Vishnu, Anar, Apoorva and the three guests from the IT company joined a teacher a group of 30 children for an amazing two hour experience. As Vishnu and Apoorva, with months of ASSET-linked curiosity bottled up, started interviewing the kids, one after another amazing moment unfolded. Children were thinking, talking, enjoying, and learning while we slowly got a glimpse into their minds, discovering misconceptions and hidden gems. Sadly, only one films was ever made from these interviews...but it was a runaway hit! The Perimeter Problem in which Vishnu brought together a triangular island, a tsunami and a goat to create magic.
The project came through....and the rest, as they say, is history!
Subscribe to:
Posts (Atom)