The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]

The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf](blendmylearning.files.wordpress.com)

42 points by aundumla 14 years ago | 10 comments

EzGraphs 14 years ago |

To Quote Martin Fowler

"Often when you come across something new it can be a good idea to overuse it in order to find out it's boundaries. This is a quite reasonable learning strategy. It's also why people always tend to overuse new technologies or techniques in the early days. People often criticize this but it's a natural part of learning. "

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#...

It is good to see some studies being done in this area (even if the one in the article is not particularly rigorous). I am convinced that there is tremendous value in the new online opportunities (Khan Academy, Stanford, Coursera, Udacity, etc), but there are limits to what they can accomplish. For one thing, an online course being done individually requires a higher level of student discipline than a learning activity involving the personal interaction of a teacher. My mind is less likely to wander when a teacher is staring me down :).

Maxious 14 years ago | |

As mentioned in the paper, there are Khan Academy "coaching" tools for monitoring the students. The site allows you to monitor the relative progress of the students so if one started to slow down you could pick them up on that: http://www.khanacademy.org/images/class-report-medium.png

EzGraphs 14 years ago | | |

Good point. This does imply that the teacher is watching the monitor rather than helping another student and then actively intervenes. So it is likely a different dynamic than a classroom where a teacher is observing the whole class at once and can take a somewhat more subtle action (i.e. speak louder in a direction of a student who is falling asleep). Khan academy may well be much more effective. It would be great to see studies that make clear comparisons of where it is most effective.

perlgeek 14 years ago |

The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers.

The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).

dicroce 14 years ago | |

Deep down in the report it is revealed that that 5.2% consists of a third of the class improving by 10% or more... and the rest remaining essentially flat. So it appears to me that it's particularly effective for a subset of learners.

Also note: the teacher doesn't have to really lead the class. The report says that the teacher spent most of their time 1 on 1 helping students who were stuck. Teachers may see value in simply having to do less work to achieve the same (or slightly better) result.

Also, this class was only a couple months. There is a question whether the small improvement would have continued and would have become wider when applied over a whole school year.

ippisl 14 years ago | | |

The class only took 5 weeks (24X2 hours sessions).

ippisl 14 years ago | |

6.4/5.2 = 1.23 .

Is 23% improvement considered that small(assuming it's statistically significant) ?

pessimizer 14 years ago | | |

I think you're looking at the wrong derivative. This is 23% improvement of improvement. It's still just 6.4% improvement.

Might be easier to see if you think of it as 106.6/105.2, in order to include the baseline.

kiba 14 years ago | |

The Khan Academy group spent 50% of their time on basic mathematical skills while the control group spend all their time on algebra. That might be a reason as to why we don't see better improvement in their score.