Introduction: the joint university sector Framework Implementation Network/Bologna Experts colloquium on supporting the design of discipline-specific learning outcomes
A summary follows of the issues raised, and views expressed, by the discipline-specific workshop groups at the joint university sector Framework Implementation Network / Bologna Experts colloquium on supporting the design of discipline-specific learning outcomes held February 2009, mentioned previously.[1]
Participants were divided into four workshop groups, one each for Business, English, Music and Physics. Each group was asked to consider and discuss the following questions:
(i) What would you describe as the learning characteristics necessary for graduates in your discipline?
(ii) How do these characteristics differ between the Honours Bachelor and Masters degrees?
(iii) Based on your response(s) to (i) above, how do you identify, in learning outcomes terms, the academic milestones in a programme?
(iv) Based on your group’s discussion, what issues arise and/or what observations would your group make in relation to:
- finding the optimum balance between discipline-specific and generic knowledge, skills and competences?
- working with the award-level descriptors and sub-strands of the NFQ?
A central question for the workshop groups was the nature of discipline-specific curriculum and the relationship between curriculum and learning characteristics (or graduate attributes). It was evident that there are clear differences in approach between the subjects in terms of identifying the ‘core knowledge base’ in a subject for a Bachelor programme. The Tuning group for Physics had found that the content of Bachelor programmes across Europe was very similar, reflecting a broad consensus about what constitutes ‘essential’ knowledge in the discipline and the primacy of that discipline-specific knowledge in the curriculum. This view was borne out in the discussions of the Physics workshop group. In the case of Music, it was found that certain core skills and competences would be considered essential in Bachelor programmes, though there may be considerable divergence between programmes in terms of emphasis (performance or academic) and repertoire. In English and Business Studies, the curriculum for a given degree programme may vary significantly from other programmes in the same subject and at the same level, even within a region.
Factors influencing the disciplinary knowledge-base for a given programme are varied and complex. They derive both from local specifics, such as the type of institution and its role in its locality and the number and research interests of individual members of the teaching staff, as well as from wider national historical and cultural contingencies. The requirements of external professional accrediting bodies are also a significant factor: formerly they tended to specify programme content, though increasingly they are specifying the graduate attributes required for professional registration, attributes which have to be expressed in terms of programme learning outcomes and achieved through stated learning outcomes for the constituent modules.
The workshop group discussions suggest that, for academics in each of the four disciplines selected, curriculum content was considered to be extremely important, but that the particular balance in learning characteristics or graduate attributes, between discipline-specific knowledge, discipline-specific skills and competences and ‘generic’ competences, might vary according to the essential nature of the discipline.
The discussions engaged in by the workshop groups are summarised below under discipline headings. Only the questions listed above that were discussed within the working groups are detailed below.
On learning characteristics:
Participants in the workshop group recognized the core competences proposed by the Tuning report on Business and the QAA benchmark statement on Business and Management at the Honours Bachelor level, which are critical thinking, analysis and synthesis; communication and inter-personal skills, problem-solving and decision-making; numeracy and planning skills; and leadership ability. It was suggested that the toolkit of a business graduate was not a conceptual one. A further characteristic identified by participants in the workshop group was the ability of graduates to develop their own ethical standpoint when faced with conflicting frameworks. Ethical responsibility was considered to be an important part of business education at all levels.
On the difference in learning characteristics between the Honours Bachelor (level 8) and Masters (level 9) levels:
While clearly there is great diversity in the range of Business Studies programmes available at the Bachelor (NFQ level 8) and Masters (NFQ level 9) level, the Tuning work on Business found that there were significant similarities in European third-level institutions regarding programme aims and content and stated subject-specific competences in Bachelor programmes, but less homogeneity at the Masters’ level. NFQ Level 9 programmes tend to focus on particular aspects of business, such as human resources management, organisational management, international business, and so on, and on the application of theoretical and practice frameworks to specific ‘real-life’ situations and problems.
On the optimum balance between discipline-specific and generic knowledge, skills and competencies:
While a knowledge of the social sciences provides a foundation for business studies, the ability to communicate effectively through oral presentations and the ability to manage and lead a project were considered extremely important in a graduate’s capacity to develop their learning in the field of business and beyond in the context of societal needs.
Ethical behaviour, analytical skills and critical thinking, developed within the context of business education, are increasingly being recognised as essential dimensions of business education
Common concerns and difficulties:
Difficulties identified during the workshop group discussion included how to represent and measure 'emotional intelligence' and 'ethical standpoint' in learning outcomes.
On learning characteristics:
Participants in the discussion in relation to English pointed to the enormous breadth in their discipline and ‘changing notions about the literary canon’. This echoed what Professor Jay had referred to as “Englishes” in her plenary address about subject benchmarking in the UK. The different characteristics of degree programmes in English in the UK derive from different programme structures, different departmental/school/faculty structures and different disciplinary contexts. It was noted that, in identifying knowledge outcomes for the graduate, the benchmarking group had to take a ‘broad brush’ approach: graduates could be expected to be able to discuss a “substantial number of authors of different periods” which might include “the period before 1800”. In this way, the subject benchmark statement for English seeks to accommodate curricular diversity rather than to prescribe a core curriculum.
Professor Jay also referred to emerging tensions between the traditional academic emphasis and the growing popularity of creative writing programmes, which further complicate the definition of knowledge-based outcomes. A common concern among UK academics is that the outcomes-based approach to higher education risks being driven by an employers’ ‘skills agenda’ towards more uniform, generic outcomes.
Another feature of the study of English highlighted in discussion was the prominence – in some programmes, centrality - of literary theory or ‘perspectives’, such as feminist or postcolonial perspectives on texts. The requirement for students to recognise and work within these theoretical frameworks or ‘modes of reading’ was a distinct dimension to be represented in learning outcomes dealing with both knowledge and competences.
There was a widely-held view that most students entering third-level programmes in English in Ireland would have a good knowledge of at least some areas of the subject and a proficiency in reading prose, poetry and play texts. With regard to the learning characteristics of Cycle 1 (Bachelor) and NFQ level 7 and 8 graduates, the most important were considered to be: ability to recognise and apply different perspectives; analytical skills; the ability to engage in self-directed learning; and the ability to present well-structured narrative and argument in written and oral formats. Arguably, with the exception of the former, these could also be considered as generic skills. What is more difficult to locate and to define in terms of outcomes is the notion of personal creativity. This may be an expected graduate attribute in creative writing programmes, but what about the academic Honours Bachelors programme? Is it a standard of individual student performance that can only be recognised and measured in terms of marking criteria? Or is the ability, in the final ‘honours’ year, to undertake independent, though closely supervised, work (for example in an undergraduate research dissertation) an indication of a creative engagement with the subject which can be assessed and represented in terms of a learning outcome?
On the difference in learning characteristics between the Honours Bachelor (level 8) and Masters (level 9) levels:
There tends to be more homogeneity in the subject background of entrants to a Bachelor programme than to a Masters programme. The capacity for independent learning and “self-assembly of relevant material” is developed during the Bachelor programme, and is essential at the Masters level. The nature of Masters programmes (evidenced in the smaller credit volume) is of greater specificity, usually within one area of the subject. A Bachelor programme provides a broad subject map, but the student on the Masters programme must gauge the potential for pushing out the boundaries of the map. In other words, Bachelor students are concerned with acquiring a broad knowledge of the subject, and Masters students with achieving a deeper, more focused and creative engagement with their material.
On identifying, in terms of learning outcomes, the academic milestones in an Honours Bachelor programme:
Foundation knowledge and skills should be developed in the early stages of a programme and be demonstrable as learning outcomes, for example by the end of Year 1 in a full-time programme over three to four years. These outcomes might include the ability to: recognise and discuss certain genres and literary forms; develop a coherent argument in the form of a written essay; and analyse some texts.
Learning outcomes must be demonstrable and capable of being assessed, and should help students to see the objectives of a given level and understand how one level builds on the other. The final year of a programme should offer a ‘vantage point’ to encourage reflective synthesis.
On the optimum balance between discipline-specific and generic knowledge, skills and competencies:
In terms of an outcome such as critical ability, students of English should be able not only to critique a specific text, but also to critique texts in general.
Common concerns and difficulties:
Some general concerns and difficulties were voiced in the workshop group discussion: it was expressed that there may be difficulty for some in distinguishing between skills and competences, and in some instances, between competences and knowledge, as outlined in the NFQ architecture. It was perceived that an outcomes-based framework pre-supposes a staged linear cognitive development and may not reflect the reality of a student’s development within a subject; and for some, learning outcomes remain prescriptive and reductive.
On learning characteristics:
As with English, participants stressed the breadth of their discipline and the differing emphases of performance-based and musicology-based programmes. There is also a professional dimension to this subject in the areas of performance and/or teaching.
A broad knowledge of a range of musical styles and music from different periods was considered to be an essential element of any degree programme, but many of the required discipline-specific characteristics for graduates are essentially non-verbal competences: musical literacy, ability to analyse a musical score, listening skills, compositional technique, etc. Graduates of performance programmes also have to demonstrate specific instrumental competences and performance technique. Music technology – which has rapidly become a prominent area in the subject – requires very specific technical, as well as musical skills. Graduates of music education programmes are additionally expected to have knowledge in the history, philosophy and psychology of education, along with effective communication and inter-personal skills. Personal creativity is important in composition and in terms of expressivity in performance. The more generic skills, such as the ability to engage in self-directed learning and research, to present well-structured narrative and argument in written and oral formats, and to engage in socio-historical reflection are also important in Music degrees.
On the difference in learning characteristics between the Honours Bachelor (NFQ level 8) and Masters (NFQ level 9) levels:
The main difference is that at the Masters level there is greater specialisation within the subject.
On identifying, in terms of learning outcomes, the academic milestones in an Honours Bachelor Degree programme:
The student progresses from acquiring broadly-based knowledge and skills in the subject to developing more widely applicable or generic skills, though these are developed in and shaped by the subject context. The skills developed in the programme are essentially the same skills in both early and late stages, though the complexity increases over the course of the programme. There are recognisable points of transition during the programme in terms of a student’s skills base.
Common concerns and difficulties:
Some general concerns and difficulties were voiced in the workshop group discussion: the question of how to represent tacit knowledge and non-verbal communication and expressivity in terms of learning outcomes was raised. It was expressed that learning outcomes represent a short-term piece-meal accountability that is detrimental to the educational process; and it was felt that an understanding of what the ‘music profession’ requires of music graduates is important in creating effective learning outcomes. However, the profession itself is very disparate and has no one representative body.
On learning characteristics:
As observed by both Tuning and the QAA benchmark statement for Physics, the Bachelors curriculum in Physics is more standardised in so far as it is based on a consensus about a significant volume of ‘core’ ‘hard’ discipline-specific knowledge that a graduate in the subject is expected to have acquired. As alluded to by Professor Jones in his presentation,[2] this can present a problem as knowledge advances and expands. By focusing on graduate attributes, rather than the detail of course content, it should be possible to avoid overloading the syllabus. An interesting finding by the Tuning group was that in continental Europe the subject had a more theoretical emphasis, whereas in the UK and Ireland the emphasis was more on experimental Physics, reflecting different intellectual traditions. The ability to solve scientific problems can be expressed in terms of discipline-specific competences, though problem-solving can also constitute a generic competence (See for example the common set of programme outcomes for the Honours Bachelor in Engineering degree (B.A.I.) used by all third-level institutions in Ireland, validated by Engineers Ireland).[3]
The highly detailed NFQ architecture of knowledge, skills and competences clearly posed problems for academics primarily concerned with the content and structure of curriculum. Discussion touched on the question of how to represent the NFQ categories of context, role and insight in relation to a graduate in Physics, and how to measure such outcomes. It was argued that context and role could be expressed in terms of competence in problem-solving, but insight, as with creativity in the context of the other subjects, was more difficult to define and represent in terms of learning outcomes. Perhaps the NFQ “insight” is what Professor Jones referred to as “deep understanding”. The argument that the generality and perceived abstraction of NFQ terminology could only take on meaning in a specific disciplinary context resonated with members of this workshop group.
On identifying, in terms of learning outcomes, the academic milestones in an Honours Bachelor Degree (NFQ level 8) programme:
This question was not discussed in any great detail, but two points of note were made.
- At the Bachelor level in Physics, in common with other sciences, the curriculum is structured around the sequential building of discipline-specific knowledge; and
- It is hoped that the student will achieve a kind of breakthrough in their understanding of the subject, what Professor Jones referred to as "deep understanding", something more than the simple accumulation of subject-specific knowledge, but developing out of a structured formation in the subject. This breakthrough may mark the passage between the Bachelor and Masters level.
On the optimum balance between discipline-specific and generic knowledge, skills and competencies:
As mentioned above, the knowledge base of the subject is growing all the time, and this presents a very real problem for defining the Bachelor curriculum: the tendency is to ‘crowd’ the syllabus rather than omit developments or core knowledge in certain aspects of the subject. Learning outcomes are a bridge between teaching and learning, and are therefore important in the design both of curricula and in teaching and assessment methodology.
Common concerns and difficulties:
Some general concerns and difficulties were voiced in the workshop group discussion: Concern was expressed about finding the appropriate balance, in an environment which requires learning outcomes on the one hand and promotes the ‘knowledge economy’ on the other, between discipline-specific knowledge and generic competences. Some considered that it is easier to define learning outcomes at the programme level than to assess the extent to which they are being achieved at the module level. It was felt that writing outcomes for programmes (i.e. single student cohorts) is more straightforward than writing learning outcomes for constituent modules, which may be taken by multiple cohorts, some on inter-disciplinary or multi-disciplinary programmes. The view was also expressed that learning outcomes statements do not assist in determining standards.
As well as the dialogue that welcomes and supports the use of learning outcomes in higher education environments, the academic world has also voiced well-documented and widely-quoted concerns, many of which were voiced in the seminar discussions and case studies presented above. Other concerns often cited by critics of the outcomes-based approach to teaching and learning include:
(i) By focusing teaching on the achievement of specific outcomes for students, the use of learning outcomes militates against students interacting autonomously with the course material, with the result that intended learning outcomes may not be achieved, though other academically valid outcomes may emerge;
(ii) Stated learning outcomes encourage students to work only towards achieving the basic threshold assessment requirements associated with a programme of study (the tick-box mentality), and may also encourage a blame culture or litigious reaction from students who are deemed not to have achieved the stated intended outcomes;
(iii) Learning outcomes are not sufficiently sensitive to the differences and specific requirements of different disciplines;
(iv) It is merely a bureaucratic exercise reflecting a system which conceives of education as a commodity, promoted by managers who do not understand the academic process; an instrument of the contemporary ‘quality culture’ which appears concerned with the lowest common denominator;
(v) Learning outcomes necessarily lead to over-assessment of students; and
(vi) It represents a ‘dumbing down’ of higher education by devaluing discipline-specific knowledge in the curriculum and over-emphasising the acquisition of generic skills.
In partial response to these and similar concerns, a number of broad benefits in the use of a learning-outcomes approach can likewise be identified. Some of these are mentioned in the seminar discussions and case studies presented above. A number of others can be outlined as follows:
(i) Emphasis on what and how a student learns
It is often argued that one of the crucial benefits of the learning outcomes approach is, that in shifting the educational focus from teaching to learning, (without ignoring the requirements of the former but emphasising more clearly the impact of course design, teaching and assessment methodologies on the latter), students’ engagement in active learning may be deepened such that they take more responsibility for their own learning. This “deep approach”, as opposed to a “surface approach” to learning may “narrow the gap” between the more and the less academically able students (Biggs, 1999). [4]
(ii) Clarity and coherence in programme design
Learning outcomes are statements of the knowledge, competencies and orientations which are formally accredited to the student upon successful completion of a programme of study; they make clear what learning is designed to take place. A direct correspondence between module and programme outcomes, supported by the underlying alignment (Biggs’ ‘constructive alignment’)[5] between content and teaching and assessment methods, leads to improved programme design. This clarity is valuable:
- for students, by contextualising their studies towards explicit outcomes;
- for teachers, by providing an articulated bridge between their teaching and assessment methods and their students’ learning;
- for external examiners, by demonstrating how the providing academic department/school is attempting to ensure coherence between module and programme outcomes;
- for employers, by identifying key skills and competences they can expect from graduates;
- for professional bodies, by assuring that essential outcomes are being met;
- for providing institutions, by enabling them to align their programmes/awards at the appropriate level on qualifications frameworks, to provide assurance as to the coherence and integrity of their programmes, and to differentiate and promote the particular emphases of their programmes;
- for prospective students seeking to enter or re-enter formal education or transfer academic credit to another institution; and
- for the functioning of qualifications frameworks and to inform internal and external quality reviews.
(iii) The facilitation of pedagogical dialogue among teachers and learners in a discipline
Making clear how and what learning outcomes are relevant to what programmes requires a high degree of mutual adjustment, communication and interaction between teachers of a particular programme, or more usually, across a set of inter-related programmes which draw on common modules. The introduction of learning outcomes in an institution is best approached not as an administrative or paper exercise, but rather as an academic process in which the collective engagement of teachers within disciplines is supported both at the local discipline or school level and at institutional level. It is this discussion that locates ownership of the process with the teachers and programme designers, and that arguably represents the most useful and fertile dimension of the learning outcomes approach to programme and module development and delivery in higher education.
(iv) Quality and comparability
By specifying learning outcomes for programmes and modules within any discipline, it is also argued that an improved degree of coherence between curriculum content and teaching and assessment methodology can be achieved, resulting in higher quality and greater comparability between programmes of study in the different subject areas. This quality and comparability is in the interests both of the higher education system and of the individual learner.
As anyone who is involved in the process of introducing learning outcomes in a higher education institution will recognise, the adoption of the outcomes-based approach to teaching and learning right across the third-level sector poses a major challenge to academics because it requires “a paradigm change” [6] on their part – or, as it is often described, exchanging the traditional ‘input-based’ or teacher-based model of university education (which focuses on course content, duration, and the lecturer’s aims and objectives) for one which focuses on students’ learning. This is not simply a question of pedagogy. Many academics, at least initially, perceive learning outcomes as undermining the intrinsic value of knowledge, of inviting a shallow, mechanistic, quantitative response from students in place of the creative intellectual engagement, based on knowledge and broad reading, they seek to foster in their students and which they consider essential to the development of their subject.
There are many practical problems too to be overcome. For example, much of the literature on learning outcomes and qualifications frameworks focuses on designing programmes such that they are consistent with this or that, whereas – especially at the Bachelor level - in reality each institution will typically have a pre-existing and complex set of inter-connecting single subject, two-subject and multi-disciplinary degree programmes which have evolved in the most economical way possible to respond to the particular local context – institutional tradition, role and disciplinary base; profile and number of academic staff in the various disciplines; student demand and marketability of programmes; professional body or industrial partner requirements, etc. On the whole, these approaches have served students, universities and society as a whole very well. Furthermore, identifying the programme may not be entirely straightforward. The Irish university system is characterised by a wide range of programmes, allowing for different approaches to framing their programme outcomes:
(a) single discipline;
(b) joint-honors;
(c) programmes comprising three disciplines one or two of which may be subsidiary;
(d) common entry programmes offering a number of different subject specialisms; and
(e) professional/vocational training programmes.
The subject-specific statements developed by the Tuning Project and the QAA provide a useful framework for single-discipline programmes. But, in the case of the popular joint honour or Arts degrees, should separate programme outcomes be written for every degree combination that includes French, or should subject outcomes be written for French and separately for each of the subject it combines with? Or should overarching programme outcomes be written for the Arts degree without reference to a particular subject? Different institutions may take a different approach, but the point to be made here is that it is a not insignificant practical issue on the ground.
Where programmes are accredited by professional bodies (or produce graduates for recognised but non-regulated professions) which have not themselves developed statements of discipline-specific knowledge and competences required in terms of graduate outcomes, writing learning outcomes is also problematic.
Another common concern about working with learning outcomes is the bureaucratic burden they represent. This presents a real challenge to institutions which are required to satisfy formal external accreditation and quality assurance requirements, while at the same time recognising and fostering the dynamic quality of teaching and learning.
[1] See Appendix 1 for a full list of speakers in the plenary sessions, workgroup facilitators and rapporteurs.
[2] Jones, G. (2009) Supporting the design of discipline-specific learning outcomes: Experiences of the Tuning Group for Physics. Paper presented at the university sector Framework Implementation Network / Bologna Experts Colloquium, Supporting the Design of Discipline Specific Learning Outcomes, Dublin 6th Feb. 2009. [Internet]. Available from:
<http://www.nfqnetwrok.ie/News/Default.76.html>
[4] Biggs, J. (1999) Teaching for Quality Learning at University. Buckingham: SRHE and Open University Press.
[6] Adam, S. (2008) Learning Outcomes, Current Developments in Europe: Update on the Issues and Applications of Learning Outcomes Associated with the Bologna Process, Paper presented at UK Bologna seminar 1-2 July, Heriott-Wyatt University, Edinburgh