Note: I originally wrote this post as a draft for
5PH1NX:
5tudent Peer Heuristic for 1Nformation Xchange
(A slightly transmedia use case in peeragogical assessment.)
David Preston, Ph.D.
_________________________________________________________________
Over the last several decades technology has driven massive shifts in the way we communicate and collaborate. Information technology, socioeconomic trends, an increasingly complex and uncertain future, and school’s failed brand are contributing factors in an emerging discourse that seeks to align learning with our rapidly changing culture. Open Source Learning and Peeragogy, two emerging theoretical frameworks in this discourse, leverage end-to-end user principles of communication technology to facilitate peers learning together and teaching each other. In both traditional and liminal learning communities, oneof the major points of contact between education and societal culture is the purposeful use of assessment. The processes of giving, receiving, and applying constructive critique makes learners better thinkers, innovators, motivators, collaborators, coworkers, friends, relatives, spouses, teammates, and neighbors. Implementing peer-based assessment can be problematic in schooling institutions where evaluative authority is traditionally conflated with hierarchical authority, and where economic and political influences have focused attention on summative, quantitative, standardized measurement of learning and intelligence. This is the story of how one learning community is adopting Open Source Learning and Peeragogical principles to decentralize and enrich the assessment process.
_________________________________________________________________
Knowledge is acquired when we succeed in fitting a new experience into the system of concepts based upon our old experiences. Understanding comes when we liberate ourselves from the old and so make possible a direct, unmediated contact with the new, the mystery, moment by moment, of our existence.
-Aldous Huxley, Knowledge & Understanding (1952)
Background
A growing tide of popular and academic attention heralds the promise of education technology. The problem is that tools and strategies such as MOOCs, videos, virtual environments, and games are only as good as the contexts in which they are used. Even the most adept practitioners quickly discover that pressing emerging technology and culture into the shape of yesterday’s curricular and instructional models amounts to little more than Skinner’s Box 2.0.
So what is to be done? How can we use emerging tools and culture to deliver such an amazing individual and collaborative experience that it shatters expectations and helps students forget they’re in school long enough to fall in love with learning again?
A world in which work looks more like this
…requires a learning environment that looks more like this
and this.
Education in the Information Age should enable learners to find, analyze, evaluate, curate, and act on the best available information. Pursuing an interdisciplinary path of inquiry in an interest-based community doesn’t just facilitate the acquisition of factual knowledge (which has a limited half-life). The process brings learners closer to understanding their own habits of mind and gives them practice and an identity in the culture they’ll be expected to join after they graduate. This requires new literacies and a curriculum that emphasizes mental fitness, physical fitness, spiritual fitness, civic fitness, and technological fitness.
Models of
assessment that emphasize self-directed
Paragogical
and collaborative
Peeragogical
principles enrich the learning experience and accelerate and amplify
deep understanding.
Because these approaches
are pull-based and generate tens of thousands of multi/transmedia data points per
learner, they generate multi-dimensional portraits of learner development and provide feedback that goes far beyond strengths and weaknesses in content retention.
The long-term benefit is exponential. Learners who can intentionally direct their
own concentration are empowered far beyond knowledge acquisition or skill
mastery.
They become more effective thinkers
and–because they are vested– more caring people.
This
learning experience is of their own making: it isn’t
business, it’s personal. The inspiration
to recreate the process for themselves and for others is the wellspring of the
lifelong learner.
As Benjamin Disraeli put it, “In general the most successful man in life is the man who has the best information.” It is a widely accepted truism in business that better data leads to better decisions. We now have the ability to generate, aggregate, analyze, and evaluate much richer data sets that can help us learn more about helping each other learn. Sharing different kinds of data in different ways will have the same game-changing effect in learning that it has in professional
baseball,
basketball, and
investment banking.
Self-directed,
collaborative assessment generates an unprecedented quantity and variety of data that illuminates aspects of learning, instruction, and overall systemic efficacy. Even a cursory examination of readily available freeware metrics, blog/social media content, and time stamps can provide valuable insight into an individual’s working process and differentiate learners in a network.
In the larger
scheme of things, Peeragogical assessment provides direct access to and practice in
the culture learners will be expected to join when they complete their course
of study. Collaboration, delegation,
facilitating conversations, and other highly valued skills are developed in plain view, where they can be critiqued and validated by peers, experts and the public.
____________________________________________________________________
SAMPLE EXERCISE: Flash Mob Mind Map
Learners responded to the
assignment with
real-time
commentary and created a
finished
product in 24 hours. The map also attracted contributors outside the original learning community who lent perspectives on content, design and working process. Although there was no formal assessment process, it was easy for each viewer to see exactly who did what and how well.
_____________________________________________________________________
But tall trees
don’t grow by themselves in the desert.
Peeragogical
innovation can be challenging in organizational cultures that prioritize control
and standardization; as
Senge et al have
observed, the system doesn’t evaluate quality when dealing with the unfamiliar, it just pushes back. In schools this is so typical that it doesn’t merit comment in traditional media. The world notices when
Syria goes dark, but in school restricted online access is business as usual.
Cultural constraints can make early adopters in technology-based Peeragogy seem like
Promethean
risk-takers.
1
Learners are not fooled by the rhetoric of in loco parentis or vision statements that emphasize “safe, nurturing learning environments.” With notable exceptions, today’s school leaders do not know as much about technology as the young people for whom they assume responsibility. Still, learners understand survival: they are fighting in unfavorable terrain against an enemy of great power. Innovating is impossible and even loudly criticizing school or advocating for change is a risk. As a result many do
just enough to satisfy requirements without getting involved enough to attract attention. Some
have also internalized the critical voices of authority or the failure of the formal experience as evidence
of their own inability: I’m just not any good
at math.
How do we know when we’re really good at something? Standardized testing feedback doesn’t help learners improve. Most of
us don’t have a natural talent for offering or accepting
criticism.
And yet, as
Wole
Soyinka put it, “The greatest threat to freedom is the absence of criticism.”
Peeragogical interaction requires refining relational and topical critique, as well as skills in other “meta” literacies, including but not limited to critical thinking, collaboration, conflict
resolution, decision-making, mindfulness, patience and compassion.
Interpersonal learning skills are
undervalued in today’s schooling paradigm. Consequently there is an operational lack of
incentive for teachers and learners to devote time and energy, particularly
when it carries a perceived cost in achievement on tests that determine
financial allocations and job security.
2Nevertheless, some
educators are introducing peer-to-peer network language and even introducing peer-based assessment.
But the contracts, syllabi and letters to
students stink of the old ways.
These one-to-many documents are presented by agents of the
institution endowed with the power to reward or punish.
To many students this does not represent a
choice or a real opportunity to hack the learning experience.
They suspect manipulation and they wait for
the other shoe to drop.
Learners also don’t like to be told they’re free while being forced to operate within tight constraints.
Consider
this
reaction to
a
policy that is
highly
regarded in the field:
“Students may choose to reblog their
work in a public place or on their own blogs, but do so at their own
risk.” What? Did I read that correctly? “Students may choose to
reblog their work in a public place or on their own blogs, but do so at their
own risk.” Risk? What risk? The risk of possibly helping someone understand
something that they didn’t before or get a different opinion than they had
before? Someone please help me make sense of this.
To effectively adopt
Peeragogical assessment in the schooling context, the community must construct a
new understanding of how the members in the network relate to one another
independent of their roles in the surrounding social or hierarchical systems. This requires trust, which in school requires
significant suspension of disbelief, which–and this is the hard part– requires actual substantive, structural change in the learning transaction. This is the defining characteristic of Open Source Learning: as the network grows, changes composition, and changes purpose, it also changes the direction and content of the learning experience. Every network member can introduce new ideas, ask questions, and contribute resources than refine and redirect the process.
This isn’t easy. A member in this network must forget
what she knows about school in order to test the boundaries of learning that shape her relationship to content, peers, and expert sources of information and feedback. This is how the cogs in the machine become the liminal heroes who redesign it. Having rejected the old way, they must now
create the rituals that will come to define the new. They are following in the path of Oedipus,
who took on the inscrutable and intimidating Sphinx, solved the riddle that
had killed others who tried, and ushered out the old belief systems to pave the
way for the Gods of Olympus.
Imagine if Oedipus
had the Internet.
Enter 5PH1NX
On Monday, April
2, 2011, students in three English classes at a California public high school discovered
anomalies in
the
day’s entry on their course blog.
3 The date was wrong and the journal topic was this:
In The Principles of Psychology
(1890), William James wrote, “The faculty of voluntarily bringing back a
wandering attention, over and over again, is the very root of judgment, character
and will. No one is compos sui if he have it not. An education which should
improve this faculty would be the education par excellence.” How have your
experiences in this course helped you focus your attention? What do you still
need to work on? What elements of the following text (from Haruki Murakami’s 1Q84)
draw your attention and help you construct meaning?The
driver nodded and took the money. “Would you like a receipt?”
“No need. And keep the change.”
“Thanks very much,” he said. “Be careful, it
looks windy out there. Don’t slip.”
“I’ll be careful,” Aomame said.
“And
also,” the driver
said, facing the
mirror, “please remember: things are not what they seem.”
Things are not what they seem, Aomame repeated mentally. “What do
you mean by that?” she asked with knitted brows.
The driver chose his words carefully: “It’s just that you’re about to do something out of the
ordinary. Am I right? People do not ordinarily climb down the emergency
stairs of the Metropolitan Expressway in the middle of the day– especially
women.”
“I suppose you’re right.”
“Right. And after you do something like that, the everyday look of
things might seem to change a little. Things may look different
to you than they did before. I’ve had that experience myself. But don’t let appearances fool
you. There’s always only one reality.”
Find
the jokers.
The jokers were
real4 and hidden (without much intent to conceal) around the classroom and in students’ journals. Students
found them and asked questions about the letters in blue; the questions went
unanswered. Some thought it was just
another of their teacher’s wild hair ideas.
Although they didn’t know it yet they were playing the liminal role that
Oedipus originated in mythology. Solving
the riddle would enable them to usher out an old way of thinking and introduce
the new.
The old way. An
authority figure sets the rules, packages the information for a passive
audience, and unilaterally evaluates each learner’s performance.
In that context, peeragogical assessment might
be introduced with a theoretical framework, a rubric, and a
lesson plan with
input, checks for understanding, and guided practice as a foundation for
independent work.
The new way. In Open Source Learning the learner pursues a path of inquiry within
communities that function as end-to-end user networks. Each individual begins her learning with a
question and pursues answers through an interdisciplinary course of study that
emphasizes multiple modalities and the five Fs: mental Fitness, physical
Fitness, spiritual Fitness, civic Fitness, and technological Fitness. Learners collaborate with mentors and receive
feedback from experts, community-based peers, and the public. They are the heroes of learning journeys.Heroes don’t respond to syllabi. They respond to calls to adventure.
Open Source Learning prepares
students for the unforeseen.
By the time
they met the 5PH1NX students had learned about habits of mind, operating
schema, digital culture and community, self-expression, collaboration, free
play, autonomy, confidence/trust/risk, and resilience.
These ideas had been reinforced through
nonfiction
articles and literary selections such as
Montaigne’s
Essays,
Plato’s
Allegory
of the Cave, Bukowski’s
Laughing
Heart, Shakespeare’s
Hamlet,
Sartre’s
No
Exit and others.
The first poem
assigned in the course was Bukowski’s “Laughing Heart”: The Gods will give you chances.
Know them. Take them. So it is with knowledge and
understanding. Today we are presented
with an overwhelming, unprecedented quantity and variety of data in our
physical and virtual lives; to cope we must improve the ways we seek, select,
curate, analyze, evaluate, and act on information.
On the back of
each Joker card was a
QR code
that linked to
a blog page with riddles and
clues to a search.
At this point
students realized they were playing a game.
A tab on the blog page labeled “
The Law” laid out the rules of
engagement:
This is
The Law.
1. You cannot “obey” or “break” The Law. You
can only make good decisions or bad decisions.2. Good decisions lead to positive outcomes.3. Bad decisions lead to suffering.4. Success requires humanity.5. “For the strength of the Pack is the Wolf, and the strength of the Wolf
is the Pack.” -Rudyard Kipling6. “The Way of the sage is to act but not to compete.” -Lao Tzu
7. Be honorable.
8. Have fun.
9. Question.
10. Sapere aude.
This is The Law.
After a second set
of on-campus and
blog quests,
students noticed a shift in 5PH1NX.
A
couple of weeks before the first clue was published, during a Socratic seminar on
Derrida’s
concept of Free Play, a student said, “We learn best when adults take away
the crutches and there is no safety net.”
The quote was used in the next clue; students began to realize that the
game was not pre-determined.
5PH1NX was
evolving in response to their contributions.
The
student’s comment was a call to action.
The Feats of Wisdom
were designed to engage learners over a vacation break in fun, collaborative,
social media-friendly missions that required engagement in the community,
expansion of their personal learning networks, and documentation on their blogs. For example:
Feat #1:
Buy a ticket to “The Hunger Games” (or any other
movie that’s likely to draw a large, young, rowdy audience). Before the
lights dim and the trailers begin, walk to the screen, turn to the audience,
and in a loud, clear voice, recite the “To be, or not to be…”
soliloquy from Hamlet (don’t worry if you make a couple mistakes, just be sure
you make it all the way to, “Be all my sins remembered.”).
Capture the event on video & post it to your blog.
Students
had been using the Internet without an Acceptable Use Policy all year; such policies are one-to-many artifacts of a central authority and far weaker than community norms. So rather than introduce “rules”
5PH1NX simply provided a reminder of the client-side responsibility:
The Emergence of
Peeragogical Assessment
If
you see something that was done with love, that pushed the boundaries, set the
standard, broke the mold, pushed the envelope, raised the bar, blew the doors
off, or rocked in some previously unspecified way, please bring it to the
attention of the tribe by posting a link to it [here].
No one did.
Instead, they started doing
something more effective. They started
building. One student hacked the entire
game and then created her own
version.
Other students began to
consider the implications for identifying and rewarding greatness. They realized that one teacher couldn’t
possibly observe how 96 students were working over vacation out in the community
and online to accomplish the Feats of Wisdom. In order to get credit for their
efforts they would have to curate and share their work process and
product. They also realized that the same
logic applied to learning and coursework in general; after all, even the most
engaged, conscientious teacher only sees a high school or college student a few
hours a week in artificial conditions.
The learner presumably spends her whole life in the company of her own brain. Who is the more qualified reporting
authority?
With these thoughts in mind
students created Project
Infinity, a peer-to-peer assessment platform through which students could
independently assign value to those thoughts and activities they deemed
worthy. Because the 2011-12 5PH1NX was a
three-week exercise in gamification, Project Infinity quickly evolved to
include collaborative
working groups and coursework. This was
learner-centered Peeragogical assessment in action; learners identified a need
and an opportunity, they built a tool for the purpose, they managed it
themselves, and they leveraged it in a meaningful way to support student
achievement in the core curriculum.
Project Infinity 2 & Implications
for the Future
Alumni from
the Class of 2012 felt such a strong positive connection to their experience in
Open Source Learning and Peeragogical assessment that they built a version for
the Class of 2013. They created Project Infinity2 with
enhanced functionality, they asked the teacher to embed an associated Twitter
feed on the course blog, and they came to classes to speak with current
students about their experiences.
Everyone thought the Class of 2013 would stand on the shoulders of
giants and adopt the platform with similar enthusiasm.
They were
wrong.
Students understood
the concept and politely contributed suggestions for credit, but it quickly
became evident that they weren’t enthusiastic.
Submissions decreased and finally the Project Infinity2 Twitter feed
disappeared from the course blog. Learners’ blogs and
project work (here
and here)
suggested that they were mastering the core curriculum and meta concepts, and they
appeared generally excited about Open Source Learning overall. So why weren’t they more excited about the
idea of assessing themselves and each other?
Because
Project Infinity wasn’t theirs. They
didn’t get to build it. It was handed to
them in the same way that a syllabus is handed to them. No matter how innovative or
effective it might be, Project Infinity was just another tool designed by
someone else to get students to do something they weren’t sure they wanted/needed
to do in the first place.
Timing may be a factor. Last year’s students didn’t meet 5PH1NX until the first week in
April, well into the spring semester.
This year’s cohort started everything faster and met 5PH1NX in November. Now (in January) they understand the true
potential of their situation and they’re taking the reins. As students realized what was happening with
the clues and QR codes they approached the teacher and last year’s alumni with
a request: Let Us In. They don’t just
want to design learning materials or creatively demonstrate mastery, they want to chart their own course and build
the vehicle/s for taking the trip. Alumni and students are becoming Virtual TAs who will start the formal peer-to-peer advising and grading process. In the Spring Semester all students will be asked to prepare a statement of goals/intentions, and they will be informed that the traditional teacher will be responsible for no more than 30% of their grade. The rest will come from a community of peers, experts and members of the public.
On Tuesday of Finals Week 5PH1NX went from five players to two hundred. Sophomores and freshman have jumped into the
fray and hacked/solved one of the blog clues before seniors did. Members of the Open Source Learning cohort
have also identified opportunities to enrich and expand 5PH1NX. A series of conversations about in-person
retreats and the alumni community led to students wanting to create a massively
multiple player learning cohort. Imagine
50,000-100,000 learners collaborating and sharing information on a quest to
pass an exam—by solving a game that leads them to a “Learning Man Festival” in
the Summer of 2013.
When 5PH1NX
players return from Winter Break in January they will transform their roles
relative to the game and the course.
Several have already shared “AHA!” moments in which they discovered ways
to share ideas and encourage collaboration and peer assessment. They have identified Virtual Teaching
Assistant candidates, who will be coached by alumni, and they have plans to
provide peer-based assessment for their online work. They are also now actively engaged in taking more control over the
collaboration process itself.
On the last day of the semester, a post-finals
throwaway day of 30-minute class sessions that administrators put on the
calendar to collect Average Daily Attendance money, hardly anyone came to campus. Open Source Learning students were all there. They have separated the experience of
learning from the temporal, spatial, and cultural constraints of school. They understand how democracy works: those
who participate make the decisions.
No one knows how this ends, but the outcome of Peeragogical assessment is not a score; it is learners who demonstrate their thinking progress and
mastery through social production and peer-based critique. This community’s approach to learning and
assessment has prepared its members for a complex and uncertain future by moving
them from a world of probability to a world of possibility. As one student put it in a video entitled “We Are Superman,” “What we are doing now may seem small, but we are part of something so much bigger than we think. What does this prove? It proves everything; it proves that it’s possible.”
