• Abstract
• Methodology
• Section A: Models for working in a discipline-specific context
• Section B: Case Studies in Writing Learning Outcomes for Discipline-Specific Programmes
• Section C: Supporting the Design of Discipline-Specific Learning Outcomes
• Final reflections
• Appendices

ABSTRACT

Qualifications frameworks provide overarching reference points to encourage consistency in, and facilitate comparability across, a wide range of educational awards. These reference points provide indicators as to the level and type of an award, and often, the volume of student workload associated with the particular award. The National Framework of Qualifications (NFQ) level indicators are expressed in terms of knowledge, skills and competences, each of which is further defined in sub-strands, for example breadth and kind of knowledge, range and selectivity of skill-sets, and context for the development of competencies[1].   In order to be relevant across the full spectrum of awards within a given educational system, the learning outcomes underpinning such frameworks are necessarily written at a high level of generality. However, since the programmes or courses leading to certified awards are invariably located within a particular field-of-study context (which may be single-discipline, inter-disciplinary or multi-disciplinary), in practice the Framework provisions become meaningful and verifiable through the articulation of discipline-specific programme learning outcomes. The programme learning outcomes can be said to define the ‘profile’ of the qualification.  This term is explained in a Council of Europe document:

The ‘profile’ of a programme/award “can refer either to the specific (subject) field(s) of learning of a qualification or to the broader aggregation of clusters of qualifications from different fields that share a common emphasis or purpose.[2] 

It is the process of articulation of programme learning outcomes, and the challenges that it presents for programme designers and teachers, that are the focus here.

Part two of the university Framework Implementation Network (FIN) report seeks to identify and explore issues arising both for individual academics and subject communities in writing learning outcomes for discipline-specific programmes that are included in the NFQ; and to look at some practical ways of addressing those issues and concerns.

METHODOLOGY

At the time this group embarked on its work, some of the institutions in which members were based had already developed learning outcomes (though in most cases at the module level only), while others had not yet begun formally to work with learning outcomes. It was anticipated that, due to differences in institutional orientations, the process adopted in each institution regarding the development of learning outcomes would be quite different, with some working from the ‘bottom up’ to calibrate existing module outcomes by level and then moving on to programme outcomes, and others starting with programmes and progressing to outcomes at the module level. It was recognised that the introduction of an outcomes-based approach to higher education in Ireland requires the embedding on the ground of a different conceptual framework, based on the idea of ‘competences’. According to the National Qualifications Authority of Ireland:

"The Framework is designed to bring about change. It introduces a new approach to the meaning of an award, that an award will recognise learning outcomes - what a person with an award knows, can do and understands - rather than time spent on a programme".[1]

For the individual academic, who is responsible for teaching his/her subject, the requirement to adopt an outcomes-based approach to teaching - to think in terms of what competences their students will have upon successful completion of a course rather than what they wish their students to know at the end of the course - can represent a very radical change, the full extent of which often only becomes apparent as one begins to engage in writing and using learning outcomes. Even where the benefits of learning outcomes are recognised, concerns persist regarding the displacement of subject-specific knowledge by generic competences and the potential for a consequential ‘dumbing down’ of higher education.  On the other hand, there is also a risk that academics will not ‘own’ the generation of learning outcomes within their disciplines, thus potentially turning the process into a ‘paper exercise’, which subsequently does not influence teacher behaviour or realize the potential benefits of an outcomes-based approach.

This working group considered that it could be beneficial to look at this particular area of tension around learning outcomes, and through this focus, to encourage more positive engagement and a sense of ‘ownership’ on the part of academic staff dubious about the benefits or usefulness of learning outcomes or even hostile to the concept of the outcomes-based approach to higher education.

In order to explore whether (and in what way) distinct approaches to the writing of learning outcomes for programmes of study might be appropriate for different academic subjects or fields of study, this group chose to look at four subjects which are widely taught across the Irish third-level education system in single-discipline and inter-disciplinary formats, and which span the arts, social sciences, ‘hard sciences’ and performance-based fields of study. These were Business Studies, English, Music, and Physics.

One of the practical problems encountered by this FIN working group was that individual institutions tend not to make their programme outcomes available externally. The dearth of concrete examples of programme outcomes in the different subject areas was frustrating. So the working group invited academic colleagues working in the selected subject areas to collaborate in a series of programme learning outcomes case studies. These are presented in Section B.

In the UK, which has been working with learning outcomes for some time, and across Europe, where the outcomes-based Framework for Qualifications in the European Higher Education Area (the ‘Bologna Framework’) and the European Qualifications Framework (EQF) are now in operation, a great deal of work has been undertaken by groups of academics working in the different academic disciplines to describe the nature and extent of their particular subject or discipline, and to define the characteristics of degree programmes so as to provide a set of representative reference points for academic programmes at the different levels. The resulting UK Quality Assurance Agency (QAA) subject benchmark statements[2][3] do not seek to prescribe the content of study programmes, but to facilitate comparability of programmes of study, while accommodating the complexity and diversity of degree programmes. The Tuning Project motto is “Tuning of educational structures and programmes on the basis of diversity and autonomy”.  and Tuning Educational Structures in Europe (“Tuning Project”) subject reports

Of the subjects selected by the FIN working group examining the topic of discipline-specific learning outcomes, all four are represented in the UK Quality Assurance Agency (QAA) subject benchmark statements at Honours Bachelor degree level, though only Business & Management has so far been treated at Masters level. Subject-specific Tuning reports have been published for Business and Physics, and the implications of the Bologna Process for the study of Music at third level is currently a key topic for the Erasmus Thematic Network, Polifonia.[4]

A seminar on the Bologna Process hosted by the Higher Education Authority, in association with the National Qualifications Authority of Ireland[5] in February 2009 in Dublin provided an opportunity to explore and discuss the views and experiences of academics currently involved in drafting and working with learning outcomes in these subject areas. Plenary session presentations from Professor Elisabeth Jay[6] on subject benchmarking in the UK for English, from Professor Gareth Jones[7] on the Tuning process and Physics benchmarks, from Dr. Peter Cullen of the Higher Education and Training Awards Council (HETAC) on the experience of developing subject-specific standards in that sector, and from Dr. Norma Ryan, Director of Quality Promotion Unit, University College Cork and Bologna Expert on linked quality assurance issues, offered valuable insights into the subject-benchmarking and Tuning processes. Four workshops took place, each focusing on one of Business Studies, English, Music and Physics. Participants explored questions concerning the learning characteristics for graduates in the given discipline, how to identify in learning outcomes terms the academic milestones in a programme, and questions concerning the desired balance between discipline-specific and ‘generic’ skills and competences. A summary of the discussions that took place is provided in Section C. 

APPENDICES 

• APPENDIX 1: JOINT UNIVERSITY SECTOR FRAMEWORK IMPLEMENTATION NETWORK AND BOLOGNA EXPERT COLLOQUIUM: SUPPORTING THE DESIGN OF DISCIPLINE-SPECIFIC LEARNING OUTCOMES

• APPENDIX 2: RESOURCES

APPENDIX 1

JOINT UNIVERSITY SECTOR FRAMEWORK IMPLEMENTATION NETWORK AND BOLOGNA EXPERT COLLOQUIUM: SUPPORTING THE DESIGN OF DISCIPLINE-SPECIFIC LEARNING OUTCOMES

Hosted by the Higher Education Authority (HEA) and National Qualifications Authority of Ireland (NQAI) on Friday 6^th February 2009, Alexander Hotel, Dublin 2.

Speakers at plenary session:

• Professor John Scattergood, Chair of Framework Implementation Network

Introduction to the university-sector Framework Implementation Network and the discipline-specific learning outcomes working group

• Professor Gareth Jones, Emeritus Professor and Senior Research Fellow in Physics, Imperial College London and Tuning Expert

Supporting the Design of Discipline-Specific Learning Outcomes: Experiences of the Tuning Group for Physics.

• Professor Elisabeth Jay, Associate Dean (Academic) of the School of Arts and Humanities, Oxford Brookes University, and member of Review Group for the QAA Subject Benchmark Statement for English

Experiences from the QAA in the field of English.

• Dr. Peter Cullen, Head of Standards, Research and Policy Development, Higher Education Training and Awards Council (HETAC)

The HETAC experience in setting award standards for the development of programmes for inclusion in the National Framework of Qualifications.

• Dr. Norma Ryan, Director of Quality Promotion Unit, UCC and Bologna Expert

How can/should quality assurance feature in the design of discipline-specific learning outcomes?

Work Group Facilitators and Raporteurs:

Business Studies

Facilitator: Mr. Patrick McCabe, School of Business, Trinity College Dublin, and Irish member on Tuning in Business

Raporteur: Professor Bairbre Redmond, Deputy Registrar for Teaching and Learning, UCD and Bologna Expert

English

Facilitator: Professor Elisabeth Jay, Associate Dean (Academic) of the School of Arts and Humanities, Oxford Brookes University, and member of Review Group for the QAA Subject Benchmark Statement for English

Raporteur: Dr. Brendan McCormack, Registrar, IT Sligo and Bologna Expert.

Music

Facilitator: Professor Jan Smaczny, Hamilton Harty Professor of Music, Queens University Belfast

Raporteur: Ms. June Hosford, Director St. Nicholas Montessori College and Bologna Expert

Physics

Facilitator: Dr. Eamonn Cunningham, School of Physical Sciences, Dublin City University and Irish member of Tuning in Physics

Raporteur: Frank McMahon, Director of Academic Affair, Dublin Institute of Technology and Bologna Expert

APPENDIX 2: RESOURCES

1. Quality Assurance Agency (QAA) subject benchmark statements:

 “Subject benchmark statements set out expectations about standards of degrees in a range of subject areas. They describe what gives a discipline its coherence and identity, and define what can be expected of a graduate in terms of the abilities and skills needed to develop understanding or competence in the subject.”[1]

Business

http://qaa.ac.uk/academicinfrastructure/benchmark/statements/ GeneralBusinessManagement.pdf

English

http://qaa.ac.uk/academicinfrastructure/benchmark/statements/English07.pdf

Music

http://qaa.ac.uk/academicinfrastructure/benchmark/statements/Music08.pdf

Physics

http://qaa.ac.uk/academicinfrastructure/benchmark/statements/Physics08.pdf

2. Tuning Project Subject Statements 

Please see Tuning Educational Structures in europe (2007) General Brochure: Introduction. [Internet]: Available from: <<http://www.tuning.unideusto.org/tuningeu/>>

Business

http://tuning.unideusto.org/tuningeu/index.php?option=content&task=view&id=96&Itemid=123

Music

http://tuning.unideusto.org/tuningeu/index.php?option=content&task=view&id=194&Itemid=222

Physics

http://tuning.unideusto.org/tuningeu/index.php?option=content&task=view&id=114&Itemid=141

 3. Guides from Irish Institutions

UCD

This guide provides examples of learning taxonomies which cover cognitive, affective and psychomotor domains:

http://www.ucd.ie/t4cms/taxonomies3.pdf

TCD

Scattergood, J. (2008) Writing learning outcomes at programme and module level. [Internet]. Available from:

https://www.tcd.ie/vpcao/bd/pdf/Scattergood_2008_Writing_Learning_ Outcomes_at_Programme_and_Module_Levels.pdf

Module Descriptor Template:

https://www.tcd.ie/vp-cao/bd/moduledescriptortemplate.php           

UCC

Kennedy, D. (2007) Writing and Using Learning Outcomes: A Practical Guide. Cork: UCC Quality Promotion Unit

NUI Galway:

This link provides access to a quick guide to writing module learning outcomes and a short video introduction to learning outcomes:

http://www.nuigalway.ie/celt/teaching_and_learning/outcomes.html

4. Learning Taxonomies

This section provides links to resources on learning taxonomies which may be helpful in constructing learning outcomes at module level. 

This resource provides a concise summary of learning taxonomies starting from Bloom’s taxonomy (1956) which focussed mainly on the cognitive domain, and includes revisions to that model:

http://www.learningandteaching.info/learning/bloomtax.htm

This resource gives a more detailed overview of taxonomies starting with Bloom, and provides a good description of the taxonomies which deal with the affective domain (attitudes & beliefs) and psychomotor (skills). It could assist with the articulation of outcomes which address communication, IT skills, performance or language fluency for example:

http://www.businessballs.com/bloomstaxonomyoflearningdomains.htm#bloom's%20taxonomy%20overview

SOLO (Structure of Observed Learning Outcomes) Taxonomy: This taxonomy developed by Biggs & Collins (1982) describes how students’ outcomes of learning display increasing structural complexity. It is a useful taxonomy for defining learning outcomes, and also for assessing the level of student learning:

http://www.learningandteaching.info/learning/solo.htm

This link gives a general overview of the SOLO Taxonomy. It shows how using the SOLO Taxonomy can encourage the development of students’ higher order critical skills:

http://www.tki.org.nz/r/assessment/atol_online/ppt/solo-taxonomy.ppt

This link from the University of Queensland illustrates the implications of SOLO for assessment design:

http://www.tedi.uq.edu.au/downloads/Biggs_Solo.pdf

This link from Southern Cross University provides guidance on how to align teaching and learning activities with outcomes using SOLO

http://www.scu.edu.au/services/tl/pathways/teaching/teaching4.html

Krathwohl’s Taxonomy of Affective Domain

This is the best known taxonomy of the affective domain and it is based on the principle of internalisation, the lowest level being general awareness of an object to the highest level characterisation where a set of values have been internalised:

http://classweb.gmu.edu/ndabbagh/Resources/Resources2/krathstax.htm

5. Other

Adam, S. (2008) Learning Outcomes Current Developments in Europe: Update on the Issues and Applications of Learning Outcomes Associated with the Bologna Process. Presentation at Bologna Seminar at Herriot-Watt University, Edinburgh 

Bergen, S. (2007) Qualifications – Introduction to a concept. Council of Europe higher education series 6. Strasbourg: Council of Europe.

Biggs, J. (1999) Teaching for Quality Learning at University. Buckingham: SRHE and Open University Press.