Sometimes it just gets to me.
Learning Upgrade may have fabulous products that produce wondrous outcomes for students (I do not know whether the company does), but I’m not buying those products on the basis of testimonial evidence. The Web site’s blog offers testimonial statements (e.g., “Reading Upgrade Helps Texas 3rd Graders Avoid Retention” and “Teenager with Learning Difficulties Makes 3 Years Growth With Comprehension Upgrade“) that are no more than anecdotal evidence.
To be sure, there is a section on research supporting the products. There are 12 studies cited (some are called “trials in schools”). How many of them would pass muster as providing strong evidence of effectiveness?
| Title | Notes |
| University Study: 1.2 Year Gain In 10 Weeks With Comprehension Upgrade, Grade 5 | This is probably the strongest study of the batch; it includes random assignment to experimental and control conditions and a reasonable outcome measure. However, it has technical flaws (e.g., use of gain scores; apparently small effects, despite the promise of the title) that make it of modest value. |
| UC San Diego Study: 1 Year Gain In 8 Weeks With Reading Upgrade, Grade 3 | No data are available; there is a summary that reduces to assertions, only. |
| New Mexico State University Study of Adolescent Special Needs | There is what is called a “full report,” but it’s an inadequate report to make much of the research scientifically. From what I can glean, it’s a pre-test vs. post-test study (no control group) using a reasonable array of measures. Without the control group, it’s not possible to know whether comparable students would have made lesser, equivalent, or greater gains than those shown in the study. |
| Early Results Show Growth With Math Upgrade, Jr High School in South Texas Sixth Grade At Risk Group Shows Gains of Almost 1 Year Within First Hours of Math Upgrade | Only a narrative paragraph provided. |
| La Costa Canyon High School: Struggling Readers Show Gains of 2+ Years Within 8 Weeks | There is a link to a summary, which provides bullet-like descriptions of students and the outcome measure, a table showing individuals’ changes in scores, and a few paragraphs of testimonial statements from students. |
| Branson School Online, Colorado: Gains of 0.5 to 2.5 Years In 2 Months With Students Working At Home | There is a link to a summary page. It contains a few brief paragraphs of description, a graph showing changes from pre-test to post-test, and 10-11 paragraphs of testimonial statements. |
| SL High School Arizona: Gains of 1 to 3 Years In Reading With Special Needs/ESL High School Students | Only a paragraph of is provided. |
| Chatsworth High School LAUSD: Special Needs Students Show Rapid Gains | Only a paragraph and a bar graph are provided. |
| ESEAL High School, Pensacola Florida: One Grade Level Improvement in 8 Weeks | Only a paragraph and a bar graph are provided. |
| Banyan Tree Day School: Success With Learning Disabled Students | Only a paragraph and a bar graph are provided. |
| READ/San Diego: Adult Learners Show Dramatic Improvement In 4 Weeks | Only a paragraph, a graph, and a link to a TV story are provided. |
| Loma Verde Elementary: Pull-Out Intervention for 103 Students | Only a paragraph is provided. |
Sorry. I’m ready to drop my $$ on these products. If they’re as good as the developers seem to think that they are, why not do the work to show it? Fund trials handled by independent research teams.
Less sizzle. More steak.
JohnL:
Thank you for taking the time to analyze the research behind Learning Upgrade’s courses. You are correct that we need more studies to confirm the effectiveness of our courses. You end with “Why not do the work to show it? Fund trials handled by independent research teams.”
So, to shed light on this situation, I will address why we don’t have more independent research, such as controlled studies, addressing the effectiveness of our courses.
Learning Upgrade is a private, family-owned business. We are not venture funded or public. There are basically three sources we could tap to fund trials:
Federal funding: we have spent quite a bit of time trying to find and apply for NIH and DofEd funding, to no avail. Their grants tend to go to the close circle of friends of the funding staff of these agencies. Evaluators tend to frown on resources like ours, which use songs, video and games. Our courses are too unconventional.
University funding: Researchers at universities want to work on their own projects. They have little interest in confirming the work of a private firm. Unless we go to them with large amounts of our own funding, they are not interested.
Our own funding: We are financed solely by the sales of our courses. If we spent the large amount of money (typically in the hundreds of thousands of dollars) to do a controlled study large enough for validity, something else would suffer. Namely, we wouldn’t have the funds to create MORE innovative online courses.
The primary goal of our organization is to help struggling students make learning breakthroughs in reading and math. We want to help each student find success. And, we want to help the millions of students who are struggling through scalable, practical solutions that work in the real world.
Thus, we are faced with a dilemma. Should we create more innovative courses, to address perhaps writing or algebra? These courses would help many more struggling students reach their full potential. Teachers and parents ask, beg, and even plead for us to produce new courses to meet their needs almost every day.
Or should we self-fund expensive research into our existing courses, which are already used successfully by over 1,000 schools worldwide, with over 55,000 students?
Perhaps you can suggest a solution. It’s not as easy as the simple statement “Fund trials handled by independent research teams.” Perhaps this works for resources created either by large corporations or taxpayer-funded universities, but not for small private organizations.
I leave you with this thought: What if the next great learning breakthrough comes from a small independent group? Perhaps teachers, perhaps working “open source” for free in their spare time, or perhaps self-funded like Learning Upgrade?
Will the world pass on this breakthrough, even though they “Teach Effectively”, just because they do not have access to taxpayer or corporate money to fund research acceptable to academics?
Best wishes and again thank you taking the time to look at our research.
Vinod Lobo, CEO, Learning Upgrade http://www.learningupgrade.com