Showing posts with label plausibility test. Show all posts
Showing posts with label plausibility test. Show all posts

Monday, December 12, 2011

Three ways of examining brain claims Part Three - learning styles

In Parts One and Two I examined the 10 percent brain myth and the educational program 'Brain Gym'. I applied the LMGTFY test and the Plausibility test to these claims (which they respectively failed). In this final post I will look at learning styles.
The term ‘‘learning styles'' refers to the concept that individuals differ in regard to what mode of instruction or study is most effective for them. of learning style typically ask people to evaluate what sort of information presentation they prefer (e.g., words versus pictures versus speech) and/or what kind of mental activity they find most engaging or congenial (e.g., analysis versus listening), although assessment instruments are extremely diverse. The most common—but not the only—hypothesis about the instructional relevance of learning styles is the meshing hypothesis, according to which instruction is best provided in a format that matches the preferences of the learner (e.g., for a ‘‘visual learner,'' emphasizing visual presentation of information). (Pashler, H. et al. 2009 - pdf)
As with Brain Gym, when you google learning styles, it is not immediately dismissed or accepted. Moreover, at first glance its claims seem more than reasonable.


There are many variants of learning styles, for the sake of this examination I'll look at the most popular model, Visual-Auditory-Kinasthetic (VAK).

Let's consider this model and its claims a bit more closely. From http://www.ldpride.net/learningstyles.MI.htm:
Visual Learners:
These learners need to see the teacher's body language and facial expression to fully understand the content of a lesson. They tend to prefer sitting at the front of the classroom to avoid visual obstructions (e.g. people's heads). They may think in pictures and learn best from visual displays including: diagrams, illustrated text books, overhead transparencies, videos, flipcharts and hand-outs. During a lecture or classroom discussion, visual learners often prefer to take detailed notes to absorb the information. 
Auditory Learners:They learn best through verbal lectures, discussions, talking things through and listening to what others have to say. Auditory learners interpret the underlying meanings of speech through listening to tone of voice, pitch, speed and other nuances. Written information may have little meaning until it is heard. These learners often benefit from reading text aloud and using a tape recorder. 
Tactile/Kinesthetic Learners:Tactile/Kinesthetic persons learn best through a hands-on approach, actively exploring the physical world around them. They may find it hard to sit still for long periods and may become distracted by their need for activity and exploration.
learning styles
I first came across VAK learning styles when I was doing my undergraduate degree in Education. I was immediately skeptical. Perhaps it was because I found I liked to learn with all three styles, and I generalized my own experience to be a universal one.... Or more likely, it relates to that expression, “If it smells like bullsh*t, it probably is bullsh*t”.

The claim was that optimal teaching and learning required diagnoses of my students' learning styles and tailoring my lessons to those individuals accordingly. I had no interest in teaching the same content in three different ways to my three different types of students based on this. When I was teaching, say, Newton's laws of motion, I varied my approaches anyway, to keep the lessons interesting and to give students every opportunity to understand a reasonably difficult concept. There were notes on the whiteboard, diagrams that I'd explain, they would conduct experiments, practice questions etc. I.e., all the students would learn about Newton's laws using all three learning styles. The mode of instruction depended on the content.

This is not to say differentiation for individual students isn't appropriate. It's the method by which we 'diagnose' who is having difficulty and why, that is important. I would diagnose which students were having difficulty and differentiate accordingly. I would do this by observation, collecting work, asking questions etc. I'd also take into account any medically diagnosed learning difficulties and act on them as appropriate.

That being said, so long as one doesn't make the claim that visual learners, for example, can only learn visually, at first glance, these claims seem plausible. People, when asked, do have preferred ways of learning. Moreover, people may be better at retaining visual, auditory or kinesthetic information. E.g. some people are natural musicians, or 3-D designers, or better at tennis.

This is the reason I think VAK seems plausible. We can all relate to being better at one style of of these or another. However, being more natural at one of the VAKs, is not the same as learning any kind of information through one of the VAKs. Conflating these two ideas is where proponents have gone wrong.

Be that as it may, let's take it as a given that the claims of VAK proponents are plausible. We now need to apply the Empirical Evidence test.

In 2009, Psychological Science in the Public Interest published an analysis of learning styles practices, evaluating their claims and reviewing the academic literature. From their summary:
A credible validation of learning-styles-based instruction requires a very particular type of experimental finding:
  • Students must be divided into groups on the basis of their learning styles.
  • Students from each group must be randomly assigned to receive one of multiple instructional methods.
  • Students must sit for a final test that is the same for all.
I.e. we need to see the results of a randomised controlled trial. If learning something in one's preferred learning style leads to improved learning, we should see that in the results. A valid learning styles hypothesis would mean students who learn with their preferred learning style should, on average, out perform students who do not.
Their literature review found that:
  • Children and adults will, if asked, express preferences about how they prefer information to be presented to them.
  • People differ in the degree to which they have some fairly specific aptitudes for different kinds of thinking and for processing different types of information.
This is pretty non-controversial. People have preferred ways of doing things. However, the literature review also found virtually no evidence for the interaction pattern mentioned above, which was judged to be a precondition for validating the educational applications of learning styles:
Although the literature on learning styles is enormous, very few studies have even used an experimental methodology capable of testing the validity of learning styles applied to education. Moreover, of those that did use an appropriate method, several found results that flatly contradict the popular meshing hypothesis.

We conclude therefore, that at present, there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice. Thus, limited education resources would better be devoted to adopting other educational practices that have a strong evidence base, of which there are an increasing number. However, given the lack of methodologically sound studies of learning styles, it would be an error to conclude that all possible versions of learning styles have been tested and found wanting; many have simply not been tested at all.
The paper, "Learning Styles - Concepts and Evidence", 2009. Association for Psychological Science. can be found here: http://www.psychologicalscience.org/journals/pspi/PSPI_9_3.pdf

Here's a good short YouTube video about learning styles.

Saturday, November 26, 2011

Three ways of examining brain (or any other) claims - Part Two - Brain Gym

In Part One I outlined the three broad tests I apply for examining a claim:
1. LMGTFY (let me google that for you)
2. The Plausibility Test (is it really even plausible?)
3. Empirical evidence (Okay, google wasn't definitive, and it sounds plausible, so what does the evidence say?)
I applied these to the 'we only use 10 percent or our brain' myth, which immediately failed the 'lmgtfy' test. This post will examine the claims made in a reasonably popular education program called 'Brain Gym'.

When you google Brian Gym, you do not get a black and white answer as to its efficacy. You get some official websites and a wikipedia entry. Brain Gym passes the lmgtfy test, so we move on to consider its plausibility.

From the aforementioned wikipedia entry:
The Brain Gym program is based on the concept that learning challenges can be overcome by carrying out certain movements, the use of which will create pathways in the brain. The repetition of the 26 Brain Gym activities (each of which takes about a minute to do), is said to "activate the brain for optimal storage and retrieval of information."
Already my skeptical hairs are beginning to stand on end. But, it's still not clear cut. Doing some form of physical activity always wakes me up and makes me feel more alert, thus 'optimizing' my brain. Exercise is good for you. No problems with that. However, there clearly is more to it than this.

It's at this point I ask 'what are they saying?' A common problem encountered when attempting to understand the reasoning behind a claim is the wording of the claim itself. A necessary condition for further investigation of Brain Gym is therefore clarity about their explanatory framework. Brain Gym is more than just 'exercise breaks'. They make explicit claims, that particular exercises effect specific regions of the brain and thus prepare students for specific types of learning. Let's consider some examples. If a picture says a thousand words, a video must say three, 'dumb, dumb and dumber'.
Okay, I've seen enough. This is just embarrassing.
Pressing "Brain Buttons" activates the brain for sending messages from the right to the left hemisphere, increases the flow of electromagnetic energy and is good for reading.
"Energy yawns" increase circulation to the brain and increases sensory perception, helping reading aloud and creative writing.  
"Hook ups" connect the body's electrical circuits for emotional centering and help students take tests and work at the keyboard. 
The claim that a specific exercise relates to improvement in a specific learning area or type of cognition is implausible. It contradicts what we know about how the brain works.
To the credit of Brain Gym, compared to say astrologers, they are specific enough that a blinded trial could easily be conducted to confirm or falsify their claims. To their discredit, as they state on the Brain Gym website, they only have anecdotal evidence and observational studies, which by definition are inherently flawed.

This post is not a critique of Brain Gym per se. This is about applying some basic skeptical principles when initially examining any claim. With a claim, such as those made with Brain Gym, we can see it immediately contradicts well established facts about how the brain works. It is, to use a technical term, pseudoscientific bullsh!t. At this point we do not need to keep on investigating Brain Gym. It is implausible; there is no need to look for any evidence of its efficacy. We can dismiss the claims of Brain Gym without much of a fuss.

In order to change my mind about Brain Gym, the known facts would need to change, or, I would need to be provided with some extraordinary evidence of its efficacy. This evidence absolutely must be more than anecdote and observational studies.

Just in case you still weren't sure that Brain Gym was a bunch of made up crap, I'll enlist the aid of Jeremy Paxman who induces this third party facepalm from Brain Gym's creator Paul Dennison "PhD". He's from California...
Occasionally you'll hear people discuss the nervous system and the brain, using the idea of circuits. Hell, I often refer to myself as a meat robot. But clearly when doing this we are speaking metaphorically. Dennison is being literal.

When describing a difficult scientific concept to lay people, it's perfectly acceptable to use analogy. However, it's unacceptable to not also describe what is actually going on. The analogy should be used for illustration only. This is even more important when being directly challenged about an explanation for a claim, as Dennison was in the clip above.

Note he also brings in the usual non-specific and meaningless term 'energy', 'midpoints' etc., all throwbacks to 'vital force' / 'life force' used by people ignorant of biology. That is, his educational pseudoscience is an extension of medical pseudoscience.

Why am I picking on Brain Gym ? Isn't it just harmless exercise? If that's all it was, that would be fine. But, it's a commercial program that schools waste good money on. More importantly, its explanatory framework is pseudoscientific bullsh*t. The job of a teacher is to tell children about how the real world works. To teach children how to think critically. Ironically, the harm being done to children's brains by this almost certainly outweighs any good the exercise does.

The clips above are from BBC's Newsnight and were sourced via Bad Science. There you will find links to the complete and unedited Newsnight clips on YouTube. For further critiques of Brain Gym see:
Bad Science 
Skeptic Dictionary

Friday, November 18, 2011

Three ways of examining brain (or any other) claims - Part One

Over the years, I've been exposed to many claims about how the brain works and how this knowledge can be applied in education. This is clearly true. Learning occurs in the brain. However, this fundamental premise does not lead to the conclusion that all claims about the brain and how it applies to education are true.

How are we to decide what to believe?

I'm only a layperson when it comes to neuroscience, but one of the good things about learning to think skeptically is being able to apply a few simple tests in order to decide whether a claim is worth believing or not. The three broad tests I apply are:
  1. LMGTFY (let me google that for you) 
  2. The Plausibility Test (is it really even plausible?)
  3. Empirical evidence (Okay, google wasn't definitive, and it sounds plausible, so what does the evidence say?)
The first two "tests" are different, but equal. Depending on the background knowledge you have about a claim, you'd pick one or the other. That is, some claims might seem plausible to people without much background knowledge. In this case a quick google might be the best course of action. Other claims would be immediately dismissed as implausible by someone with the required background knowledge, so a google search is unnecessary.

The third test only comes in to action when the first two tests are passed. If a quick google doesn't resolve the truth of the claim, or it sounds plausible, then deeper research is required. We need (high quality) empirical evidence.


If only there was some way of easily searching for information... here lmgtfy

I'm constantly amazed at the seeming inability, or unwillingness (if I'm being fair), of people to google. There are some claims that are checked with a 30 second google search. Take, for example, the factiod that we only use 10% of our brain.

Snopes (check), Wikipeda (check), Scientific American (check), and boom, myth busted. No need to move on.

I know a bit about brains, biology and evolution

As the Snopes article and the Wikipedia entry explain, the claim that we only use 10% of our brain is a biologically implausible. If I had a background in biology, I’d have already known, for example, that brain imaging shows all the brain is active, brain injuries lead to a loss of function, and evolution by natural selection is a merciless process that would eliminate such a wasteful inefficiency.

Thus my background knowledge allows me to dismiss this claim as implausible without heading to google.

These two ways of evaluating a claim are very simple, and work well for claims that have a black and white answer. Of course, many claims aren't so black and white.

In two more posts I’ll apply the same logic to claims about the brain and learning that are not as easily dismissed. Part two will look at “Brain Gym”, which lacks plausibility, but isn't immediately dismissed when doing a quick google. In Part three I will look at “Learning styles”, which again, isn't resolved with a quick google, but compared to Brain Gym, seems very plausible.

Update. Here's an example of what can happen to you if you don't check your facts...
(face, meet palm)