Learning Without Assessment and The Willie Clancy Summer School

If you're ever lucky enough to find yourself in Miltown Malbay, County Clare, Ireland, around the early part of July each year you will find a most wonderful musical and learning event taking place: the Willie Clancy Summer School. Sadly, one of its founders Muiris Ó Rócháin passed away this year. Many years ago I made a TV documentary called Up Sráid Eoin about the wren boys of Dingle and Muiris featured prominantly in the film. Ar dheis De go raibh a anam usual (trans: May he be honoured at the right hand of God) .

Now let me tell you about this school. It uses a form of cascade to facilitate musicians of all levels (and ages) to improve their playing of traditional music. The development of Irish set dancing skills is also an integral part of the week long programme. So experts teach the proficient, the proficient teach the novices etc.. In addition, there is a very strong social aspect to playing traditional music, its fullest expression is through group playing with a mixture of fiddle, flute, tin whistle, uilleann pipes, accordion, guitars, banjo and bodhráns.

Throughout the week there is a mix of structured classes and spontaneous sessions usually in the local pubs. These sessions embrace young and old, expert and novice - what we get is one continuous, joyous expression of music - all the while people are learning new tunes and developing their skills.

There is no assessment, no barriers to 'giving it a go', no penalties for mistakes and great learning takes place.

This entire approach is completely at variance with our institutionalised version of education. We often take for granted the assumption that all learning must be assessed and we see learning as an individual and often lonely process. The Willie Clancy Summer School gives us a different perspective on this.

Thanks Muiris.

Ssshhh!!! Exams in Progress!

This is a quiet but busy time in National College of Ireland semester one exams are now in progress.  We are encouraged to keep as quiet as possible as each room on the campus is now used to its fullest extent to facilitate the process.

There is always tension associated with exams.  Students of all ages and all backgrounds find the prospect of being tested daunting. This is very understandable we live in a culture of measurement and accountability and education is an expensive process.  So, especially for the self-motivated,  we all want to see how much we know and how well we have progressed.

As discussed previously, there is a useful distinction between goals and achievements that are measured by independent criteria such as exams or tests and other achievements that are socially referenced such as how other people regard performance.  

For many students the exam results provide important feedback on how they have coped with the learning challenges associated with a course.  At some point in the future the result–put simply as a number–will be revealed.  

However, this will only be a very small part of the story.  The student, and the student alone, will know how to interpret it.

Best of luck to all concerned.

Leaving Certificate Results

Today's top story is the issuing of results to almost fifty eight thousand (58,000) Leaving Certificate students.

This event is widely reported in national newspapers, radio and television news.  Much of the coverage deals with the failure rates for different subjects.  Of special interest is the success rates for Science, Engineering Technology and Mathematics- the so called STEM subjects.  It is reported that some 4,300 students have failed Mathematics.

The availability of a talented young workforce is often cited as part of the attraction of Ireland as a location for inward economic investment. Poor results do not help the international perception of our education system. Employers are increasingly looking for critical thinking, creativity and problem solving skills in their new recruits.  It is reasonable to ask to what extent the results of the Leaving Certificate Exam may be taken as an indication of a person's future potential as an innovative thinker and an effective contributor to the knowledge economy.

Take for example the substantial cohort of young people who have failed Mathematics.  Are we justified in writing off the potential of these people as college students, future inventors, knowledge workers, scientists, entrepreneurs and innovators?  I feel not.

Letter to Sam (a fictional character representative of the 4,300):

Dear Sam,

Today you got your results and I guess it came as no surprise that you are one of the 4,300 students who failed mathematics. It must have been very disconcerting to listen to media reports on the importance of Mathematics for our future economy.  Surely, you must be thinking that it will be very hard to get a job or go to college. What now are your career options and prospects for the future?

Sam I'm not going to say that that none of this counts and that the results of your Leaving Certificate are of no consequence - that's certainly not the case!  What I do say is that when you put things in perspective you have much more choice and more potential that you think.

Be very careful about accepting labels, especially labels that you give to yourself, at this stage in your life.  You may wish to say I'm no good at maths! - perhaps this is something you've always believed about yourself and now you feel vindicated, you were right all along and your Leaving Cert results prove it.  Well, that may be the case but it is also likely that other factors are in play.  Have you ever heard of a self-fulfilling prophecy?  Perhaps it was your own belief about being no good that caused you to apply little effort or energy to the subject.  Of course, once you fall behind with Maths its harder and harder to catch up. So you need to genuinely ask yourself is it that you fell behind and simply need time to catch up or do you have a more fundamental problem with Maths.

Sam what are you good at?  Is it that you are good at sport or do you know about cars, or fashion or music?  Think about how you became good.  How did you develop these skills?  It took time and persistence even tenacity, lots of practice and most especially, you were interested and believed you could progress.   This is how you became a skilled footballer (musician, mechanic, hairdresser and so on).You know other people who wish to be as good as you are and you might even say look its easy.  Well to you its easy but it may be very daunting for other people - just like Maths is for you.

So you might ask is there really anybody who is genuinely no good at Maths or is it all about the perceptions, teachers and opportunities? The answer is complex - I mean yes and no - Maths generally involves abstract thinking and many people have a generalised difficulty with this form of thinking.  The best way to explain abstract thinking is to compare it with its opposite - concrete thinking. 

Here's an example, a family of four are preparing to go on a motoring holiday and your task is to load the boot of the car.  As you might expect some people have brought two suitcases and what's more the car has a very small boot and the cases come in all shapes and sizes.  Now in order to complete the task do you start to load and move each case testing where it will fit best (concrete thinking) or do you work out a scheme in your head as to how the whole lot will fit (abstract thinking)?  In this example each approach has merit.  Some people are 'knackey' they are good with their hands and they can visualise how things will fit together.  This spacial ability is closely related to mathematical ability it is a really useful skill.  However, some people use it in concrete situations and never really seem to be able to apply it in abstract form.  In my opinion people who have good spacial ability have the potential to be good at Maths but they don't always fulfil this potential.

Regardless, Sam you need to know that you will be learning throughout your life and the setback today may be an opportunity in disguise. State exams are just one measure of the potential of an individual and the Maths exam is just one aspect of that measure.  To survive and thrive in this world we need to be intelligent in a multitude of different ways - we need language skills, social skills, kinaesthetic (movement) skills, awareness of nature, spacial skills and yes, mathematical and logical skills (see Howard Gardner's works on multiple intelligence).  Build on your strengths - society needs people with all these capacities and everyone has something to offer.

Best wishes for the future 

Leo

 

"Grade Inflation" Getting Everything Wrong

This is a really important issue for Ireland and for everyone in the education sector.  It is vital that get a clear understanding of what the problem is and what we need to do to rectify it.

First of all, the problem we need to solve is not "Grade Inflation" and it would be a huge mistake if we were all to get in a muddle comparing the numbers of first class honours' degrees or 600 point Leaving Certs in the past few years.

Just like all measures based on our social circumstances, such as the spending power of the average weekly wage or the average life-expectancy, over time we should expect to see a gradual improvement in similar measures of quality and achievement in our education system.

Today, we are educating more people to a higher standard than ever before and I will be surprised if the emperical evidence from the soon-to-be released study will not show this to be the case.

But I do not believe we should be congratulating ourselves - there is a problem and a new challenge and we need to get to the heart of it.

Let me use one source Dr Craig Barrett, former CEO and Chairman of Intel and a frequent visitor to Ireland:

 "Your primary and secondary schools are only average," he said. "It is no longer good enough to be average. You have to be excellent at what you do ... at the end of secondary school your young people are average. Your education system is being challenged by improvements in the rest of the world. Things have changed, the educational attainment of other countries have been increasing, and that increases competition for attracting investment."Source: http://www.examiner.ie/opinion/columnists/matt-cooper/for-ireland-to-make-the-grade-we-need-radical-education-reform-111903.html#ixzz0h0o2hsCx

Barrett is providing us with a global perspective and he, rightly in my opinion, points to the progress made by other countries.  Later in the same interview Barrett lays down the challenge:

"It is possible for Ireland to continue to be successful, but you have to worry about the capability of your workforce and what it does," he said. "Why not a race to the top? Why not have more capability and jobs where you can add value? Increased capability and education is where you increase value."

Later"Teachers should not be just competent but masters," he argued. "How do you excite children about a subject? By having high-quality teachers. You must treat education like any quality service you aim to provide. Your employees have to be good and get paid on the basis of performance."Source: http://www.examiner.ie/opinion/columnists/matt-cooper/for-ireland-to-make-the-grade-we-need-radical-education-reform-111903.html#ixzz0h0qGJr4J

Now, let me make plea: let's not get ourselves in a flap over grade inflation or comparisons between institutions.  Let's talk about what really matters - quality of teaching and quality of assessment.

It is a not sufficient for the Department of Education and Science to look to the State Exams Commission (note "exams" not "assessment") to produce year-on-year comparisons of Leaving Cert grades - why don't we look at what the Leaving Cert is really measuring - mostly memory, recall and strategic learning.  Genuine problem-solving and creative thinking are not nurtured and not sufficiently recognised.

Similarly, in third level we are certainly guilty of over rewarding students who do not ask questions, suggest alternatives, write critically or challenge the norms of society.

This is the real threat!  In short, it's not that we are giving too many high grades in exams, it's that we are not measuring what we should be measuring.

Certain skills are more important for competitive and connected workplaces - these include inquiry, problem solving, technical and scientific skills, critical thinking, research, collaboration, presentation and good writing.
These skills need to be nurtured and measured at all levels of education.  This is the real challenge.

Knowledge Surveys

I came across an interesting piece on Knowledge Surveys from Edward Knuhfer and Dolores Knipp (linked above).

They advocate the use of Knowledge Surveys as a tool in support of learning and instruction.
These surveys consist of a series of questions - similar to a set of exam questions - but the difference is that the learner is asked not to answer the question but to rate their own ability to respond.

For example - consider the following questions:

Q1 Describe three characteristics of an constructivist theory of learning?

Q2 Compare constructivism with social constructivism?

Q3 Outline practical applications of a behaviorist approach to learning?

Now, in a traditional assessment the student would be asked to write short essays on the above.

With a knowledge survey the student is asked to rate their level of knowledge as:

A - I feel confident that I could answer this question

B - I know about 50% of what may be involved and perhaps if I went away for twenty minutes I could find the missing information

C - I am not confident that I would be able to answer this question at all

Do you get the gist? The knowledge survey gauges a student's perception of their own ability.

Knowledge Surveys may be very useful particularly at the beginning of new courses or topics. A word of caution though - students may not always have or report a reliable estimate of their own ability.

What are we teaching in schools?

Two very interesting comment pieces appeared in today's Irish Times. The editorial commented on the draft report by the National Economic and Social Forum on the connection between school literacy levels and social exclusion and inside, a piece by Breda O'Brien (link above) on creativity and second level education. It is interesting to connect the two pieces.
As a society we have a responsibility to prepare young people for the future - this is what we expect of our education system - but we cannot possibly know what the future has in store. As the educational philosopher John Dewey put it - the best we can do is to teach children how to experience the present to its maximum extent.
Our children are poorly served by an archaic education system where state exams focus on selective recall and pure luck. Notice that we have the State Exams Commission not the 'educational assessment' commission indicating that they are only concerned with 'exams' one form of educational assessment. This is like an orchestra that can play any music as long as it is composed by Mozart!
Future oriented skills such as critical thinking, inquiry, creativity and collaboration are largely undervalued in the present school system. Until we reform the pedagogy of schooling and assessment we will continue to suffer the consequences of poor literacy levels. And large numbers of young people from disadvantaged backgrounds will continue to face a future on the margins of society.

Course Entry Requirements - Recognising Learning from Experience

If you are thinking about taking a course, for example any of the NCI courses in the prospectus, you may see in the entry requirements that it is necessary for students to have a specific level of degree (e.g. honours degree) or a certificate or diploma to gain entry.

These conditions are necessary so that all students are able to participate effectively and teaching staff can make certain assumptions about the level of prior knowledge people will have.

However, there is a down side to this in that sometimes very good potential students miss out because on paper they are not deemed to meet the entry level requirements.
We've all come across examples in our work where people with significant experience and competence in a particular field are not necessarily the most qualified in the formal academic sense.

Not many people know this but there is a mechanism whereby anyone can obtain a formal academic credit (yes I mean a degree, diploma or certificate) by means of providing evidence that they have achieved the learning outcomes equivalent to a recognised qualification.No this is not some e-mail scam to give people cheap meaningless degrees from a little known US private college - this is the policy of our own Higher Education and Training Awards Council (HETAC) and it is enshrined in the legislation used to establish this national awarding authority.

Here is where you apply to HETAC for this process.

It is now accepted that there are three contexts in which learning occurs:
formal learning - this is when you undertake a course of study usually with a view to obtaining a formal award or qualification;
non-formal learning that takes place sometimes in the workplace (e.g. training courses) or community or voluntary sectors - although often assessed it does not normally lead to formal certification
and informal learning - sometimes referred to as experiential learning and takes place through life and is often not recognised a s learning by the individual concerned. Experience is the key driver for new knowledge and the development of competence.

Educators now recognise that all three of these contexts are important sites for learning. The challenge is that accreditation bodies need formal systems to measure learning outcomes and understandably they require that potential candidates produce a portfolio of evidence which is accessed and verified by an academic panel.

To go back to the entry requirements for courses - did you know that it is possible to make a case that your extensive experience should be taken into account when apply for a course where, on paper you do not appear to meet the entry requirements.

All colleges operate such a scheme - this is especially the case in NCI where wider access to learning is our core mission.

The process requires that the applicant undergo some form of appraisal to demonstrate that they have achieved the equivalent learning outcomes as those with formal qualifications.

This may involve preparing a portfolio of experience or writing an essay or assignment to demonstrate your competence - in all events it will be evidence based.

If you really want to do the course and feel that you know more about the area through experience - you can prove your case through accreditation by prior experiential learning (APEL) - its more straightforward than you think.

So go on! What are you waiting for.

Leaving Cert English Fiasco - There Was Another Way!

Big Problem!

In assessment terms, the majority of our state exams may be characterised by unseen (in advance) questions and time limited tests.
The shock news of today is the fact that through some unfortunate human error the questions for Leaving Cert English paper 2 were inadvertently distributed to a small group of students intending to sit paper 1.
"The integrity of the exam had been compromised by the regrettable incident" said the Minister for Education Batt O'Keefe.
The State Exams Commission considered they had no option but to cancel today's paper 2 exam and ordered that a new paper 2 should be taken by students this Saturday.
This is no small inconvenience it is very distressing for the students concerned, it will cost a lot of money and it has discredited the operational effectiveness of those responsible for organising the exams.

Was there any alternative?

The simple answer is yes and it is a great shame that some lateral thinking was not applied to the problem.

The issue had to do with the consequences of some students knowing the questions one day in advance.
Let's suppose that we want the exam process to adhere to two principles that may have been undermined by the leaking of the questions in advance - the first is the 'unseen' nature of the test and the second is the principle of 'fairness' in that some students will have seen the questions and some may not.

Seen and Unseen Exam Questions

Let us deal with the consequences of students seeing the questions in advance. What if the papers were corrected with this knowledge in mind? Open book and open or seen question (i.e. the questions known in advance) exams are not at all unusual in the third level sector.
Once the person correcting the scripts knows the conditions under which the exam was taken it is simply a matter of taking this into account.
It's really no big deal that the students knew one day in advance which poets they will have to write about.
The other, much more important, issue is that of fairness. A situation where some students knew the questions and others did not would violate this principle and would be unacceptable.
The Department of Education claims that they found out about the breach of security at 4pm yesterday afternoon and had to make a decision on the resit within a very short time frame. I have some sympathy for them and someone has to answer for the fact that the error was not reported sooner.
But was that the right or only decision available?
No!
I suggest that the Department should have published English paper 2 there and then and used the news media to disseminate that fact.

In this way all students could read the paper and prepare on equal terms.

There was no ideal solution once the security of the system broke down but publishing the exam paper would certainly have been the least worst option.

Perhaps the whole fiasco will provide a stimulus for some much needed rethinking on how we assess learning at a national level.

Plagiarism Reframed

Mention plagiarism to any third level academic and you are likely to be greeted with groans and laments.
This is one topic that gets into people's hearts – it leads to animated discussions and hard views. It is unwise to be regarded as soft on the issue.
It is annoying, very annoying to be reading something presented as a student's original work when it dawns on you - this is familiar - or - this is not the same style of writing as expected.
Plagiarism is genuinely offensive to many academics - it offends one's sense of academic integrity and is regarded as a dishonourable practice and a form of cheating.
Many also feel that the student is trying to make a fool out of them – the tables are reversed - instead of the assignment being a test of the student it is a test of the examiner.
Assuming the examiner will not spot the obvious is a form of insult.

In most institutions plagiarism is treated as a disciplinary rather than learning or teaching matter - student's face expulsion, suspension and fines if they are found guilty of the charge.
Remarkably, despite clearly stated policies and warnings to students - it seems that the incidence of plagiarism is increasing rather than decreasing. All in all it is of great concern and worry.
There is a need for a radical rethink of how we conceptualise and deal with plagiarism.

Most treatments of plagiarism begin with a definition and they look to dictionaries as the source (always a worrying sign) - something like – plagiarism is the act of passing off other peoples written work as your own etc..
Much of the academic practice centres on how to spot plagiarism and how to punish it. There is a good business in the technology of plagiarism detection (most people know the Turnitin software).
Of course, as the technology on the detection side gets better – so too there are many more Internet sources to copy and even services that will write your assignments for a fee.
We have the plagiarism wars – each side trying to outwit the other. As with all wars there are casualties on both sides.

“If I was you I wouldn’t start from here at all” said the wise Kerryman when asked for directions. So with plagiarism let’s leave it for a while and come at the problem from an entirely different starting point.

A constructivist pedagogy assumes that we build new knowledge through the interaction of present and past experiences. I like to refer to this process as the act of making meaning. For example, when I read good theory the ideas resonate with me – I connect these new insights with my past experiences. An essential characteristic of the constructivist model of learning is that making meaning is a unique and personal process. There is no universal knowledge just personalised knowledge.

Dewey contrasts the traditional and constructivist approaches to learning:

On the one hand, learning is the sum total of what is known, as that is handed down by books and learned men. It is something external, an accumulation of cognitions as one might store material commodities in a warehouse. Truth exists ready-made somewhere. Study is then the process by which an individual draws on what is in storage.
On the other hand, learning means something which the individual does when he studies. It is an active, personally conducted affair.

(Dewey, 1944 p 335)

Dewey’s own vision was in keeping with the latter active notion of learning expressed above. If you agree with a constructivist model of learning (and most theorists do) then there is no such thing as purely original work. Even these words as I write are made up of insights and ideas from many sources – true I have integrated these with my own.

So for most of this text – which I claim as my own writing – I am making meaning from multiple sources from people, experiences and feelings in my past. Note that where I cite from Dewey above I indicate in the format that these are Dewey’s exact words – I am inviting you the reader to make your own meaning.

When students are given written assignments they are being asked to make meaning not to reproduce knowledge.

Plagiarism is a refusal or failure to make meaning.

There are many reasons why people refuse or fail to make meaning. Sometimes students are confused about what is expected, some students are reluctant to express their own ideas as they feel this is not real learning. Other students worry about their ability to write and are in awe of other peoples words – how could I write it better than an expert. Some cultures are more reluctant to question great writings and individual meaning making is discouraged.

And yes – some students are genuinely dishonest and are attempting to cheat the system.

How can we deal with plagiarism?
The first point is that prevention is far better than cure. Cheating is really only significant where high stakes assessment is involved. In other words when students are being ‘tested’ and the result forms part of their grade. A strategy of providing early ‘low stakes’ or formative assessment events will provide feedback to students who miss-learn what is expected of them when they write.

Secondly, academic writing requires additional skills and specialist knowledge such as how to format, cite and prepare bibliographies. As with all skills people learn best by a mix of rule learning and practice. When used properly, citations and quotations may provide a form of scaffold for the novice academic writer while he or she is finding their own voice and meaning. But many students at this stage fail to apply the citation rules and often regard them as incidental - a question of format rather than core content. Early and frequent opportunities to practice academic writing with rapid feedback on errors and progression will counter this.

Finally, what of the cheats – what’s really happening here? I believe cheating is also a consequence of miss-learning. It is a failure to learn values. The values of academic integrity and the collaborative quest of knowledge underpins the third level education system. This frequently gets mixed up with the economics of qualifications and the preparation and entry points for jobs. A student who cheats believes that there is a short cut to a qualification and that the assessment is too blunt an instrument to catch them.

This may say something about the standards and practice of assessment as well as the character of the student.

References
Dewey, J. (1944). Democracy and education (First Free Press Paperback ed.). New York: Macmillan.