What could the peer evaluation form and the portfolio look like for this course?
- Login to post comments
What could the peer evaluation form and the portfolio look like for this course?
Participants develop projects throughout the course and present to the group at the end.
The finished project would be a learning product:
- relevant to the participant's application context
- demonstrating learning objectives achieved
- peer and expert reviewed
its the scale of the assessment that's the problem. How would you get 'experts' to review 300 projects? 600? 5000?
Have an online clickable rubric that would be filled (by all) right after the presentation, aggregate the data, and return it to the presenter in the form of voted 'scores' -- insta-feedback.
I would consider a self-assessment tool that was part of an ongoing dialog with peers. The self-assessment form would include two aspects: 1) an understanding-of-content component and 2) a performance component. There would be periodic checkpoints where each participant would have an opportunity to articulate (i.e., summarize) what he or she understood and did not understand at that point and time. From there, action plans would materialize to include 1) things to begin doing, 2) things to stop doing, 3) things to do more of, and 4) things to do less of.
All of this information would need to be shared with peers throughout the learning process in order to gain perspective and alternative interpretations.
If someone needed to provide evidence of their understanding and skills obtained by taking this course, they could aggregate all their efforts (including their self-assessment form) in the form of an eportfolio.
There's my answer... :)
This is how I see the education system moving from rewarding attendance to engagement and application. I would 'morph' the items listed by bnleez into a structure guided by a reflective schema, such that provided by Jennifer Moon. This would include artifacts as suggested for this course. The end result, call it a portfolio or artifolio could be distilled into a version for addition to a personal learning environment of CV. Just a thought.
You'll note that clicking 'portfolio' will now pull together all but your comments.
Thanks! What would really be helpful for me is to have a "portfolio" (kind of a cross between what the "portfolio" and "recent posts" are right now) that lists my most recent posts (regardless if I initiated the discussion or not) so I could track my own discussions more easily. I added to a discussion yesterday (one that I did not initiate) asking about a rubric and for the life of me can't find the darn thing. It's starting to become difficult to track my discussions.
Or even a search feature would be nice. If there is a search feature and I missed it, let me know.
It's a bit tricky assessing an online community of +700 members if the main goal is to support each member reach his/her OWN objectives. Rubrics and criteria would do no good in this since they are based on clearly stated learning objectives. So one strategy that might work in this situation is to have each member post a reflection by the end of the course reflecting on their own experiences in this course, what their aim was, if they reached it or not, how they reached it, and discuss some of the main points of the course that they felt relevant to them and how they have developed an understanding of these points through this course. I think this way we can assess each member individually in light of their own previouse understanding, their goals, and how this course supported them in developing a deeper understanding of the issues.
I aslo like the idea of co-authoring papers, maybe we can split into groups and have each group agree on a topic that interests them. We can even use Google Docs as a co-authoring tool to make it easier to collaborate.
Doesn't look like you have posted a template yet, just a place for one.
Being shy of a blank page, here are some thoughts to discuss and maybe transfer
One part should address the content in a critical and creative way. That is, ability to identify and think critically about future shaping trends.
Another, identify own perspective and that of others WRT future.
Use of data to think about trends
Understand context and how it effects trends
The template would be a rubric for assessing these. Learners would collect filled out templates relative to specific contributions they make in this course
Hi Nils,
I know i haven't responded directly, but i've been following along and taking your input into account... something about having 700 people in a course seems to suck up some of my time :P
I think we're coming at this from different perspectives. I see the strong verbs in your language, and the higher level learning that they are connected to. Let me offer the opposing perspective and then we start negotiating :)
so...
I think, generally, that these things defy accurate measurement. If you structure a rubric for them, you tend to take them out of context, and if you try to identify them in context you do a fair amount of guessing.
This model presumes that one person can be an authority on another's, say, creativity. I get nervous around judging people's core abilities in an adult learner environment. It's one thing to say 'create a blog, make a blog post about monkeys and drop your url in an OPML file' and I"ll check... it's a bit different to say 'give me a creative solution to this problem'. Creativity involves moving beyond YOUR OWN boundaries...
It also suggests that everyone needs to take the same kinds of things out of the course. A middle administrator in a university and a high school music teacher would be coming to this course with different needs and should get different things out of it.
As we are not tied to any university constructs, there are no 'must haves' that articulate to the next 'more complex' course. Realistically, the best way to take this course to the next level is to take it again knowing what you learned last time.
Your other comment suggests that employers would want to evaluate rubrics... I'm working under the impression that most employers wouldn't be bothered... FOR THIS KIND OF CREDENTIAL. I would feel differently if it were, say, a plumbing course. On what basis would an employer judge the good/badness of a futures course?
_____________
One of the main issues i'm trying to deal with is scale. If we have a course on a topic where there is no defined 'canon' of knowledge, then we can, in a perfect world, negotiate it for ourselves and our contexts. Credentializing this is going to be very complex if we're measuring.
I turn the table back to you!
Dave,
I've been waiting for you to come back to this credentialling topic. We have been thinking about How learners working on the open Internet can "harvest" assessments (or feedback) back from where they are working to a central place, like their university course.
A recent post. http://communitylearning.wordpress.com/2010/03/30/engaging-employers-and...
A summery post pointing to a group of others is http://wsuctlt.wordpress.com/harvesting_gradebook
I think this would be a great group to explore the idea deeper, because one of the ideas we discuss (but don't get to explore as much) is validating the rubric with the employer. In this case, learners could take the rubric to their employer as part of getting the sign off.
"We cannot take wood from the Kings forest, nor take fat trout from the King's rivers without the proper papers and licences". Your email made me smile as we circle our way back to the crux of the matter with accreditation. We assume the Uni has more ability to determine whether a student has GRASPED the idea, when clearly, around the world - many University Lecturers are not accredited with the skill to explain what an outcome is; let alone explain the difference between formative and summative assessment. They may be experts in their field - but the plurality of the web kind of makes them not 'the' expert.
We come back to paying the 100 Gold Coins to the get paper. In feudal times, to leave or enter the King's Castle, you needed exactly the same things. A mans honesty was judged by this papers.
In these times too; the retort to this was to form a Guild. It is high time we stop messing about grovelling to the Kings and Barons and formed a movement in which the accreditation means something. To those who want this to become part of your 'work' - consider that this course and the connections made will be your new workplace, so think of it as investment opportunity. I doubt the boss will truely understand and give you papers either.
Let's get on with it, stuff the kingdom.
While I work on a longer reply to some of the ideas in this thread, here is breaking news the follow up to a story about Cathy Davidson that appeared in the Chronicle last summer (our reply to Cathy last summer http://communitylearning.wordpress.com/2009/08/14/crowd-sourcing-to-supp... ).
The breaking story:
http://www.insidehighered.com/news/2010/05/03/grading
There are comments from a number of cynics, partly because we can't see the work or the assignments that prompted the work.
I think this is relevant because those same critical voices might take a similarly skeptical view of peer assessment leading to credential in this course.
The business side may resolve this "for" us, Dave. I applied for a position last week where so many applicants were in line ahead of me that the business was issuing a hypothetical problem for each of us to solve.
We would "essay" our proposed resolution which would display our real understanding of the domain.
Actually, this has been a trend for some time from the business side. The business community has less trust in certifications by and large. Real competencies, skills, capacities, and knowledge are being tested.
In fact, what businesses want are employees who possess a certain amount of leadership skills, some interpersonal skills, and an understanding of the issues with which their position would be involved.
With the rapid changes we face in nearly every knowledge-domain, how can last year's certification be useful today? I see I can now get certified as a Social Media Marketing expert if I pay any of a dozen or more schools and the like for an online class. How did "they" become experts to the degree that they can offer certification?
Future cloud-workers will need to be adept at contributing to problem-solving communities which themselves are constantly changing with the emerging trends.
Therefore, to secure employment, I suspect a worker would need to demonstrated skill via up-to-date portfolio contributions or response to a hypothetical problem solving scenario.
I conducted 100's in the last decade for my employer, generally we asked situational questions to have applicants demonstrate their thinking and performance in the past so as to be able to predict future behavior, we did not worry too much if they understood the problem, just if they came to the correct solution. This was in the IT field, but the employee competence landscape was pretty stable and relatively predictable. Certifications proved to mean absolutely nothing as we considered employees from 14 year olds, just starting High School to adults with Master degrees in Computer Science... and hired more 14 year olds than the "educated" adults. The papers did nothing for telling the employer how effective the individual would be for a particular task, but the behavioral interview techniques did a great job of filtering out those people that would simply not fit into the team environment. The employer typically used the hypothetical problem assuming that a a particular solution was correct, this is no longer sufficient.
Now, if we add the rapidly changing knowledge/skill landscape, employers are likely to be looking for adaptability. There obviously needs to be a knowledge baseline, and the ability to collaborate with co-workers, but after that the workers ability to extrapolate into the future will be key. The portfolio is a perfect way to "WOW" a potential employer and demonstrate competence and adaptability in an ever-changing knowledge landscape while a paper credential is more a sign of mastering a static knowledge peak.
I would extend your idea to include hypothetical multiples of problem-based setting and solving within a complex adaptive system. That is, the learner, employee, etc. is able to see patterns within a given context such that problems can be both set and addressed. I say addressed because the goal would be to create multiple possible outcomes to a given problem as opposed to one single "correct" problem resolution.
In the old days of working in the creative industries, before journalist too illiterate to subscribe to Gizmodo and Mashable's feeds coined the term 'Social Media Expert' -- a designer, producer or director lived by the strength of their portfolio.
More importantly, they learned very quickly to network that talent and to court the agents that would utilise their skills and creativity.
The C.V. perhaps got you the first studio 'dogsbody job' but beyond that -- all that ever mattered were connections and reputation based on the folio.
In gaming, a players character is represented by their associations, race and class - though rep points and achievements. You cannot be a top level player without first grinding your way through the numerous quests, puzzles and challenges.
To to conclude, I believe the portfolio, should be include published research - as today it is very normal for that to be team-based, not individualised. There are hundreds of journals asking for papers constantly.
Would this be a more valuable as a demonstration of life long learning that merely ticking boxes through an Masters for example?
It would allow people to follow their passion in a way that our systems can relate to; allow people to work together to achieve it - globally - and learning the 'ropes' of research would life their ability to evidence their intellect - to show they 'get it'.
So part of this folio - should perhaps be co-published papers.
- Login to post comments
This is all so fascinating! Thanks for practical applications for alternative assessments.