Assessment for online teaching and learning
Discussion post from a professional development course for online community college teachers on assessment, grading, and feedback:
The administrators want “data” they can show to their “stakeholders” (not my favorite word) to demonstrate the “effectiveness” of teaching and learning. As you know, I take a rather … ahem … non-traditional approach to this topic, which is one of the reasons I was asked to develop this course. My own personal view is that “data” that show that students scored 80 percent on a test before, and then 90 percent on a test later, doesn’t really say much about the effectiveness of teaching and learning, unless what you’re teaching your students to learn how to take multiple-choice tests.
There are a whole series of issues here involving test validity, levels of knowledge that can be assessed, learning domains, learning styles, expectations of the discipline, accreditation requirements, learner preparation and prior knowledge, and so forth. Most administrators are not “real” educators <grin> and come from the epistemological view that quantitative data from test scores are an objective measurement of real learning. I take the view that assessing through multiple choice tests — especially in areas such as the humanities — is never truly objective, and is probably not even measuring the learning that actually happening.
We have to play the hand we’re dealt. If they want multiple choice tests, give it to them. It is very unlikely you will persuade a committee to take an alternative approach, since these tests are cheap and easy to administer, and everyone can agree that 90 is a higher score than 80 (probably the real reasons they are used). So I give them all the spreadsheets they want. Then I go about subverting the system by focusing my efforts on authentic assessment models, such as student portfolios or whatever. A funny thing happens. The administrators who really lack the imagination to see assessment as anything but a spreadsheet get an opportunity to see what authentic assessment looks like, and they love it! They want to host portfolio shows and invite their “stakeholders,” and put student videos on the campus YouTube site, along with PowerPoint presentations and photographs of smiling students with their projects, and otherwise show the world what their students can actually DO!
So that’s my strategy. Give them what they ask for, then SHOW them what my students can actually DO. I don’t know if they ever really understand it as being “real” assessment, but they usually dig it and can see the benefit to the institution (whether or not they see the benefit for the learners). I’d rather work the fringe, wiggle through the gaps, and play within whatever sandbox they give me, than try to get them to imagine something about learning they’ve probably never experienced themselves.
The line for co-conspirators forms to the left. :-)
A vision of education in the year 2000 … as imagined in the year 1910.
Villemard, 1910, À l’ École, Visions de l’an 2000
Given the power of our prior beliefs to skew how we respond to new information, one thing is becoming clear: If you want someone to accept new evidence, make sure to present it to them in a context that doesn’t trigger a defensive, emotional reaction.
Chris Mooney, The Science of Why We Don’t Believe Science, Mother Jones
Terrific research-based article with important implications for learning.
Source: Mother Jones
Creative students are teachers' "least favorite"
“Psychologists at Union College surveyed several dozen elementary school teachers in 1995. While every teacher said they wanted creative kids in their classroom, they were mistaken. In fact, when the teachers were asked to rate their students on a variety of personality measures – the list included everything from “individualistic” to “risk-seeking” to “accepting of authority” – the traits mostly closely aligned with creative thinking were also closely associated with their “least favorite” students. As the researchers note, “Judgments for the favorite student were negatively correlated with creativity; judgments for the least favorite student were positively correlated with creativity.”
OECD: Train teachers to become education researchers
“The most successful countries educationally make teaching an attractive, high status profession, and provide training for teachers to become educational innovators and researchers who have responsibility for reform. These were among findings presented last week in New York at the International Summit on the Teaching Profession, the first of its kind, held to identify best practices for recruiting, training and supporting teachers.”
Will higher ed forfeit its teaching role to for-profits?
“For-profits are better placed to expand online because they do not have to worry about resistance from academic staff, nor about exploiting their earlier investment in campus facilities. A disruptive technology, which online learning may prove to be, rarely favours existing providers,” said [President of the Commonwealth of Learning, Sir John] Daniel.
That raised the question of whether higher education would split over the coming years into a public sector focused on research and a for-profit sector doing most of the teaching. Daniel said the recent dramatic funding cuts to UK universities was likely to significantly swing the balance in funding higher education towards research and away from teaching.
I’ve been arguing this for years, but I’m not sure the bulk of the teaching will necessarily go to for-profits. There’s a great role for undergraduate-focused teaching colleges in public higher ed, but community colleges are too stretched by their multiple missions, and faculty and administration at universities are much more interested in their research role than their teaching role.
Higher education is now at a crossroads. If public higher education forfeits its teaching role to more nimble for-profits, students will be the losers.
TEDTalks: Salman Khan: Let’s use video to reinvent education (2011)
Salman Khan talks about how and why he created the remarkable Khan Academy, a carefully structured series of educational videos offering complete curricula in math and, now, other subjects. He shows the power of interactive exercises, and calls for teachers to consider flipping the traditional classroom script — give students video lectures to watch at home, and do “homework” in the classroom with the teacher available to help.
If John Dewey or Paulo Freire came back as a hedge-fund-analyst-turned-21st-century-educator, they would be Sal Khan.
Source: ted.com
What’s wrong with this picture?
(Source: rizkyselma, via world-shaker)
However neat and tidy this image may be, we really need to put this sorry old chestnut to rest, and remind ourselves of the dangers inherent in citing secondary sources.
Will Thalheimer (2006) published a rather detailed and well-cited debunking of this oft-repeated learning myth. Thalheimer traced the concept to Dale’s Cone of Experience (1946), pointing out that Dale’s image contained no percentages. The percentages came from D.G. Treichler (1967), an oil company employee writing in a magazine about audio-visual communications, but without citing any research as a basis for the percentages. An organization called NTL Institute claims to have developed the Learning Pyramid in the early 1960s. Thalheimer presents one graph that includes a citation, but the cited article contains no such graph or percentages (meaning the citation was in error, or fraudulent). Thalheimer confirmed this with the author of the article the graph cited.
A report commissioned by Cisco from the Metiri Group (2008) goes into further detail. The report provides evidence of how this myth has been widely perpetuated — and manipulated — in the fields of education and training for several decades, despite the complete absence of empirical evidence supporting it. The report then goes on to summarize genuine research into multimodal learning, deriving information from fourteen published scholarly articles (based on twenty-three studies and meta-studies that together surveyed almost 6,000 students) to determine the best practices for multimodal teaching, whether or not it includes technology.
If you need any further evidence, try a Google image search for “Dale’s Cone of Experience” to see how these images endlessly ricochet around the web when a “reblog” is just a click away. If we see it repeated often enough, from otherwise reputable web sites for educators, why shouldn’t we take it as gospel? Limit your search to college and university web site (add “site:edu” to your search), and you may see that the echo is as loud in education as elsewhere.
One of the reasons this information has been endlessly repeated is that it “feels right.” It seems to fit rather intuitively with our perceived experiences, and meshes nicely with the preference many of us have (myself included) regarding active and experiential learning through multiple modalities. Somewhere along the way someone slapped some numbers on it, and suddenly it became credible, citable science. I can only wonder how many theses, dissertations, and peer-reviewed articles in scholarly journals include this bogus information among their references, dutifully quoting a secondary source without ever checking the facts.
Ironically, this myth-busting does more than refute an old canard. It actually illuminates a very real research-based finding about learning: it is very much harder tounlearn something than to learn it correctly in the first place. Our brains, once “wired” into a familiar pattern, find it much easier to recognize and repeat the same pattern, and find it much harder to “rewire” to change the original pattern. Automaticity in learning has enormous benefits that allow the non-conscious aspects of our minds to repeat familiar patterns without taxing the precious resource of conscious thought. Yet that same automaticity can mask our perceptions and prevent us from even recognizing that the pattern we are repeating has little basis in reality.
Perhaps the bottom level of the Learning Cone should say, “People remember 100% of ideas that go unquestioned.”
Metiri Group, (2008). Multimodal learning through media: What the research says. http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-Media.pdf
Thalheimer, W. (2006). People remember 10%, 20% … oh, really? Will at Work Learning. http://www.willatworklearning.com/2006/05/people_remember.html
Source: rizkyselma
Where to begin?
Teachers greatly underestimate the significance that prior knowledge has on learning. Very little can be learned and retained effectively except when it can be connected to prior knowledge for each learner. Teachers often pick a starting point in a course or a lesson without considering what it is their students actually know before they begin, and too many fail to use any type of prior knowledge assessment to find out.
If teachers paid as much attention to prior knowledge as they do to crafting novel presentation techniques to address differing learning styles, they would marvel at what their students could actually learn.
Why are syllabi so badly written?
The federal government requires corporations to write their disclosure statements to the SEC in “Plain English.” Now if only we could get college teachers and administrators to apply the same principles to writing syllabi.
Teacher
by Audre LordeI make my children promises in wintery afternoons
like lunchtime stories
when my feet hurt from talking too much
and not enough movement except in my own
worn down at the heel shoes
except in the little circle of broken down light
I am trapped in
the intensities of my own (our) situation
where what we need and do not have
deadens us
and promises sound like destruction
white snowflakes clog the passages
drifting through the halls and corridors
while I tell stories with no ending
at lunchtime
the children’s faces bear uneasy smiles
like a heavy question
I provide the food with a frightening efficiency
the talk is free/dom meaning state
condition of being
We are elementary forces colliding in free fall.And who will say I made promises better kept in confusion
than time
grown tall and straight in a season of snow
in a harsh time of the sun that withers
who will say as they build
ice castles as noon
living the promises I made
these children
who will say
look— we have laid out the new cities
with more love than our dreams
Who will hear
freedom’s bell deaden
in the clang of the gates of prisons
where snow-men melt into darkness
unforgiving and so remembered
while the hot noon speaks in a fiery voice?How we romped through so many winters
made snowballs play at war
rubbing snow against our brown faces
and they tingled and grew bright
in the winter sun
instead of chocolate we rolled snow
over our tongues
until it melted like sugar
burning the cracks in our lips
and we shook our numbed fingers
all the way home
remembering
summer was coming.As the promises I make children
sprout like wheat from early spring’s wager
who will hear freedom
ring in the chains of promise
who will forget the curse
of the outsider
who will not recognize our season
as free
who will say
Promise corrupts
what it does not invent.
Source: wiresabuzz
Teaching seldom comes naturally
New faculty start from a stronger position when they believe that regardless of natural ability, much about teaching still needs to be learned. Some instructional knowledge is straightforward, but just beyond those first easy answers are a slew of complicated algorithms mastered with practice and a commitment to pursue excellence. For the vast majority of teachers, learning to teach and continuing to teach well requires concerted effort and plenty of good, old-fashioned work. And like most other kinds of learning, there is always more to learn. It is impossible to know everything about doing well in the classroom and with students.
— Maryellen Weimer (2010). Inspired College Teaching: A Career-Long Resource for Professional Growth
Is it true that we can only learn when we are aware of being taught?
By Eve Kosofsky Sedgewick, in Touching Feeling: Affect, Pedagogy, Performativity.
What does it mean when our cats bring small, wounded animals into the house? Most people interpret these deposits as offerings or gifts, however inaptly chosen, meant to please or propitiate us, the cats’ humans. But according to the anthropologist Elizabeth Marshall Thomas, “Cats may be assuming the role of educator when they bring prey indoors to their human owners…. A mother cat starts teaching her kittens from the moment they start following her…. Later she gives them hands-on practice by flipping victims in their direction, exactly as a cat does in play. Mother cats even bring [wounded] prey back to their nests or dens so that their homebound kittens can practice, especially if the prey is of manageable size. So perhaps cats who release living prey in our houses are trying to give us some practice, to hone our hunting skills.”
For persons involved with cats or pedagogy, Thomas’s supposition here may be unsettling in several ways. First there is the narcissistic wound. Where we had thought to be powerful or admired, quasi-parental figures to our cats, we are cast instead in the role of clumsy newborns requiring special education. Worse, we have not even learned from this education. With all the cat’s careful stage management, we seem especially stupid in having failed to so much as recognize the scene as one of pedagogy. Is it true that we can learn only when we are aware of being taught? How have we so confused the illocutionary acts of gift giving and teaching? A further speech act problem here involves imitation: the cat assumed (but how could we know?) that its own movements were templates for our mimicry, rather than meant to be made room for or graciously accepted by us. A gesture intended to evoke a symmetrical response has instead evoked a complementary one.
Then again, even if we had recognized the cat’s project as pedagogical, it’s possible we would not have responded appropriately by “honing our hunting skills” in the broken, twitching prey. Possibly we do not want to learn the lesson our cat is teaching. Here, in an affective register, is another mistake about mimesis: the cat’s assumption that we identify with it strongly enought to want to act more like it (e.g., eat live rodents). For a human educator, the cat’s unsuccessful pedagogy resonates with plenty of everyday nightmares. There are students who view their teachers’ hard work as a servile offering in their honor — a distasteful one to boot. There are other students who accept the proffered formulations gratefully, as a gift, but without thinking to mimic the process of their production. No doubt this describes a common impasse faced by psychoanalysts and psychotherapists as well. Teaching privileged undergraduates, I sometimes had a chilling imitation that while I relied on their wish to mirror me and my skills and knowledge, they were motivated instead by seeing me as a cautionary figure: what might become of them if they weren’t cool enough, sleek enough, adaptable enough to escape from the thicket of academia into the corporate world.
And beside the frustrations of the feline pedagogue are the more sobering ones of the stupid human owner. It’s so often too late when we finally recognize the “resistance” (mouse flipping) of a student/patient as a form of pedagogy aimed at us and inviting our mimesis. We may wonder afterward whether and how we could have managed to turn into the particular teacher/therapist needed by each one. Perhaps their implication has been: Try it my way — if you’re going to teach me. Or even: I have something more important to teach you than you have to teach me.
Source: books.google.com
Sal Khan keeps amazing me with his enthusiasm and brilliant analysis of new ways education can work. This talk with the MIT Club of Northern California is a bit long (1:23:28), but worth every minute of it. The potential for the future of teaching and learning is fascinating.
