Vic's Educational Software Blog

The Transference Problem.

Here's a great article by George Mason law professor Thomas Hazlett that encapsulates the objections that opponents have to teaching with computers. His is not the raving of a Luddite. Many people have good, reasoned objections to machine-based learning. Hazlett cites studies where students working on computers actually performed more poorly on standardized tests than students who worked traditionally. He's afraid that technology-enabled learning does little more than teach computer skills. He's on to something.

At Carnegie Learning, we called this the "transference" problem. We saw a lot of it in schools: students would perform well on our software, but then fail offline tests. The learnings they had made on the software didn’t “transfer” to a novel situation. This is a meta-cognitive problem for education generally. It’s difficult to teach in the controlled environment of the classroom such that the students have problem-solving capacity with novel situations in the world outside the classroom. For computer-based education, this is a particularly acute problem because the interactive environment of the computer interface is by necessity limited and controlled. If the interface is built the wrong way, the student can only solve problems with the computer scaffolding built in—they cannot solve problems offline, in novel situations. You might think of the transference problem as the difference between a student researching a paper online and then writing it on his word processor, and a student who simply assembles a paper by clicking and pasting material he has gathered online. One child learns; the other doesn’t.

It's hard to build an interface and human-computer interaction scheme that actually incorporates the student's cognition into the design. It's much easier to simply build around the student interaction. Most software, for example, simply tells students whether they got something right or wrong. This is practically useless from a teaching standpoint: the student needs to figure out what they did wrong in order to learn the things they need to do right. If your interface design doesn’t provide the space for students to do that, they will not learn.

Figuring out exactly this space is the hardest thing of all. It’s for the most part domain specific: the space for reading doesn’t translate into the space for math. It’s also grade specific: designing that space for a second-grader is different than designing that space for a seventh-grader. Most importantly, because this space is tied to the specific skill you are trying to impart to a student, how exactly you do this with interface design could be unique in every instance, and you will never get good economics out of designing a different interface for every skill in a learning program (or, for that matter, a good user experience).

Addressing this problem comes down to expertise—you need people on your staff who have designed software, repeatedly, that actually teaches students. There aren’t that many people out there who have actually done this. Many claim to. But few have. The way to discriminate between those who claim this skill and those who actually have it are to ask them how much time they’ve spent in usability with students. Those who understand good interface design because they’ve actually watched students interact with educational software (or they have instrumented their software packages in order to gather directional data) will understand the complexity of the problem. Commercial software publishers usually spend an enormous amount of time on usability and test; education publishers tend to rely on an “expert” with an advanced degree. This is not enough. The only way you develop good skill in designing effective interfaces is through direct observation in classrooms (not labs). You can’t teach this stuff in books. You have to learn it by doing.

The upshot is that producers of technology-enabled curriculum programs will only meet the objections of skeptics by providing objective, off-line assessments that validate the educational gains made by students using their software. But those students will only succeed at those tests if the interface design addresses the transference problem. Absent that, computers in the classroom will fail, and become, as Hazlett points out, the latest version of a technology, like educational TV, that failed to deliver on its promise.

# posted by vicbastard @ 5:59 AM