Discipline

Business/HR

Programme

Master of Business Studies in Human Resources Management, NFQ Level 9

College

Kemmy Business School, University of Limerick

1. Please outline the process you adopted in order to arrive at an agreed set of programme outcomes. 

Note: By ‘process’ is meant both the local drafting process, and the verification process at institutional level.

First steps included attending training sessions in the University and consulting with the Course Board. I drafted the learning outcomes for the programme taking into account any insights and input from lecturers on the programme. Following this, I sent the programme outcomes to the Course Board for any comments. It was important to incorporate learning outcomes from all modules on the programme – it was also important to clarify the level of student understanding/knowledge/skills – for the MBS in HRM, this level would be level 6 on Bloom’s taxonomy.

Verification at institutional level was through Head of Department, Dean and finally the Vice President.

2. What were your main reference points (internal and external)?

Prompts: Did you consult Tuning Group descriptors, UK subject-benchmark statements, documentation from professional institutes, NQAI documentation? 

Was there a consensus about what are the most important things that students of your discipline should learn in the selected programme (graduate attributes)?

 My main reference points were NQAI literature and the training sessions provided by the University of Limerick. However, I also consulted other programme leaders, the Course Board for the MBS and other documentation from professional institutions. The UL training sessions and the information on the Centre for Teaching and Learning website were extremely helpful. There was general consensus about what a graduate should know/do and to what level.

3.    What challenges did you encounter in the drafting stages and how did you overcome them?

The main challenge was trying to capture the programme learning outcomes within the framework given. I grappled with the problem of including generic skills and competencies while maintaining a subject specific focus, It was a challenge to effectively communicate to students the expectations and desired outcomes of the programme in a clear, economical and meaningful way. However, the training course and the NQAI literature helped with this.

4.    What have been the benefits (if any) of drafting and working with learning outcomes at programme level and at module level?

I found this to be extremely beneficial for several reasons:

-          Thinking about and drafting the programme and module outcomes provided me with an opportunity to reflect on the programme (in terms of philosophy, expectations etc)

-          It required me to think about what the essential things a student needs to know/do upon completion of the programme. The use of Bloom’s taxonomy helped with this process as it provided a hierarchy of ways students can demonstrate their understanding. This helped me clarify what a MBS student needs to know and do – and how well they need to do this.

-          It also highlighted the importance of having measurable outcomes – it is important to be able to assess the learning outcomes. I realised through this process that I had written vague and immeasureable outcomes in the past. I have also learnt through this process the importance of using the appropriate action verbs when writing learning outcomes.

5.    What has been the impact of the programme learning outcomes on teaching, assessment, and on student behaviour and performance?

It is quite difficult to fully answer this question at this stage – particularly with regard to student behaviour. However, from my own perspective (and from my conversations with some of the programme lecturers on the course, they seem to agree) I found the process has provided me with the valuable opportunity to rethink some of my assessments and learning outcomes. It has provided me with clarity and has helped me focus on the needs of the student.

In terms of the impact on students, it is too early to say but I hope this process will improve the communication between lecturers and students and will clarify for students what is expected of them and how this will be measured. I also hope it will help potential students make better informed decisions regarding what programmes they want to do.

Discipline

English Literature

Programme

B.A., NFQ Level 8

College

NUI Maynooth

1. Please outline the process you adopted in order to arrive at an agreed set of programme outcomes. 

Note: By ‘process’ is meant both the local drafting process, and the verification process at institutional level.

In 2007, the School of English embarked on a wholesale revision of its BA (omnibus entry) programme from Level 1 (Year 1) right through Level 3 (Year 3). In addition, the School introduced for the first time a new BA in English (designated entry). Programme content and outcomes were developed and agreed at programme-development meetings of the Department before being forwarded to the Teaching and Learning Committee of the University for discussion and approval; subsequently, they go before the Faculty of Arts, Celtic Studies and Philosophy for review and approval.

Initial discussions about programme content may not have been couched in the language of learning outcomes but were concerned with questions of a balance between coverage (for instance, do students need to have the traditional survey course on Romanticism?) and approach. At the core of the programme are modules that constitute a traditional English syllabus, covering English literature from the Renaissance through the Restoration to Romantic Poetry and Modern and Postmodern culture, and systematically covering the main literary genres: fiction, drama and poetry. But the importance of designing a syllabus that would seek to reflect the situation of English literature in Ireland today was also recognised. So, the programme statement mentions “As Ireland becomes both more culturally diverse, and increasingly wants to examine its own culture, this degree will allow students to study Irish writing in a global context as a strand within an English degree”. Additionally, with reference to both University and broader Government targets to increase graduate intake, it was felt that the integration of undergraduate and graduate study, with the former directly feeding into MA and research degrees, should be facilitated. This is reflected in the programme statement (“Those students who may wish to consider further postgraduate study will find that the programme provides a solid, three-year grounding in research techniques and literary theory”) and also in the content.

There was broad consensus as to the outcomes of the programme: that is, in their final year, students who studied English at NUI Maynooth would have: good communication skills; strong analytical and conceptual facilities; the capacity to identify and also to use different theoretical approaches. And, discussions worked from desired graduate attributes (critical-thinking skills; lateral thinking; good oral and written communication; self-directed work and group work) back to how these might be realized through individual modules. It was felt that increasingly students entering university are not equipped with the requisite critical thinking skills for studying literature at university level and that is unfair to expect them to absorb these, as if by osmosis.

Thus, from the outset it was felt that individual modules needed to interconnect more explicitly and facilitate the development of the student’s critical aptitude. In re-designing the programme, then, the importance of identifiable strands running through from Level 1 to Level 3 was recognised. Accordingly, Level 1 is made up of four modules Criticism and Research (I) and three Studies in Literary Form modules dealing with (II) Poetry, (III) Fiction and (IV) Drama respectively. Both the Criticism and Research strand and Literary Form strand are developed in Levels 2 and 3. The objective with the Criticism and Research module is to introduce students to the analysis of literature at university level and to orientate their learning in terms of critical-thinking skills and research methods. Through Criticism and Research II and III respectively, students engage in particular theoretical schools in these later modules, so the movement is from the general of Level 1 to specificity and depth at Levels 2 and 3. This is reflected in the learning outcomes for these later modules, which indicate a higher expectation and specificity (e.g. “On completion of this module, students will have a formation in critical and cultural theory and, through small group seminars, developed their own research topics”). The objective here was to provide a learning map that would enable the student to clearly identify expectation (from Level 1 to 2 and 3, from general to specific, survey to in-depth) and, crucially, progression. And, this is an attribute of the revised programme that the external examiners (BA 2009) identified as a particular strength.

In addition, in designing the programme it was felt that it should reflect current trends in Irish and international scholarship more explicitly – in that way, students taking English at NUI Maynooth would be able to identify how their degree was different to comparable degrees elsewhere. To this end, it was decided that we would have two strands, Irish Studies and World Literature, that would begin in Level 2 and continue into Level 3. It was additionally noted that foregrounding these areas at this stage usefully feeds into the two MA programmes (Twentieth Century Irish Writing and Culture, Empire and Postcolonialism).

2. What were your main reference points (internal and external)?

Prompts: Did you consult Tuning Group descriptors, UK subject-benchmark statements, documentation from professional institutes, NQAI documentation? 

Was there a consensus about what are the most important things that students of your discipline should learn in the selected programme (graduate attributes)?

We looked at internal programme and module learning outcome templates but for the most part the emphasis was on colleagues’ sense of trends in the discipline and also experience of teaching to a large and varied student cohort. As stated above, there was agreement as to graduate attributes and recognition of the need to write these into module objectives and assessment methods.

3. What challenges did you encounter in the drafting stages and how did you overcome them?

For a subject like English that is by its nature quiet, discursive and nuanced, module descriptor templates can present some problems as they can appear rather scientific or mechanical. Accordingly, effort was made to accommodate these templates to the subject, and to balance coverage and method (e.g. EN 353 Postmodern Writing and Culture: “On completion of this module students will have a good knowledge of the key critical accounts of postmodernism and will be experienced in using these in the interpretation of contemporary literature and the arts”) without seeming reductive or simplistic.

The importance of blending outcomes and assessment was recognized and for the most part an effective relationship between the two has been achieved with the new programme. There was discussion about varying modes of assessment and this is the case with all modules, which involve a combination of essay, examination and weekly Moodle or e-learning exercises. It was noted that small-group seminars and seminar essays facilitate the optimum blending of outcomes and assessment for a discipline like English. However, resources limit the extent of this significantly. That said, the advantage of the formal exam, where students are required to engage with unseen questions in a finite time, was noted as a very good barometer of their competencies and knowledge in the discipline. 

4. What have been the benefits (if any) of drafting and working with learning outcomes at programme level and at module level?

As a general principle, developing learning outcomes for a module is a useful exercise both for lecturers in early stages of their careers, and for those who are more experienced. For the former, it helps control the tendency to make modules overly complex or dependent on their own research interests; for more experienced lecturers, writing learning outcomes provides an occasion to reflect on the main purpose of modules that may have developed and evolved over the years.   From an administrative point of view, it is helpful to be able to look at learning outcomes for the totality of all modules in a programme as a way of getting an overall impression of the coherence and direction of a given programme. 

5. What has been the impact of the programme learning outcomes on teaching, assessment, and on student behaviour and performance?

In introducing a new programme, the result has been more cohesive and integrated modules and a clearer sense of a student’s progression through to graduation. In terms of assessment, there is a clearer sense of what each module is ‘doing’ and how it relates to the overall programme. It has been our experience to date – confirmed by the external examiners for BA 2009 – that students are drawing from different modules in their continuous assessment rather than compartmentalizing material as happened with the old syllabus. Also, with increasing consciousness about learning outcomes (although there persists among some academics a scepticism about the shift towards this vocabulary), we have implemented Grade Descriptors for all markers and for all forms of written assessment. These descriptors, which are made available to students via Moodle, indicate different levels of competency and specificity that neatly and usefully overlap with learning outcomes as they progress from Level 1 to Levels 2 and 3.

6.  Free comments

There is a need to exercise a certain amount of recognition of the specificities of individual academic disciplines. Having been involved in running both a Media Studies programme and an English programme, I would have a clear sense that in the case of the former, certain modules (those in media production) would have strongly instrumentalist outcomes: (“To enable students to use ProTools to edit...”); whereas in a Humanities discipline such as English, the learning outcome should not be forced to conform to an instrumentalist learning agenda. Sometimes, the intended learning outcome of a module on the Victorian novel is simply to learn about the Victorian novel – not to teach transferable skills relating to composition by stealth.

Finally, I would strongly urge the abolition of the distinction between “aims” and “objectives” that many module descriptors seem to require. I have yet to receive a convincing explanation of the difference between these two categories; most people I know simply cut and paste the content of the “aims” field into the field for “objectives”.

Discipline

English Literature

Programme

BA, NFQ Level 8

College

Arts and Celtic Studies, UCD

1. Please outline the process you adopted in order to arrive at an agreed set of programme outcomes. 

Note: By ‘process’ is meant both the local drafting process, and the verification process at institutional level.

These comments relate to two linked, but separate processes: (1) the formulation of learning outcomes across the whole suite of modules offered in English; and (2) the writing of learning outcomes for modules designed to be delivered using Enquiry Based Learning (EBL). In the first instance (1) a small group within the school (a subset of the teaching and learning committee) drafted some initial outcomes, trying to identify the qualities, competencies and knowledge that we wished a graduate of English to have. This was initially a fairly lengthy list, which we then attempted to place into categories – in other words moving from complex and detailed statements to short, clear, generic ones. What one might call ‘programme outcomes’ are relevant to several aspects of the programme – at module level, at each stage, and also at level 4 (Masters’ level). The movement overall is from specific to general, from detail to simplicity, so a level 1 (First Year) learning outcome might state that the student will learn how to locate an article on Chaucer’s Canterbury Tales on JSTOR, whilst a level 3/programme one will set out that the students will acquire research skills relevant to the discipline. For certain kinds of module, e.g. seminars, many of the outcomes would be held in common across modules, with variations to take account of specific subject areas – this helps to make a programme delivered through 100+ modules cohere to some degree, whilst encouraging variation. In the second case (2) the approach was radically different as the modules were defined by process and delivery rather than purely by content. Thus the usual process of developing outcomes was inverted so that the module structure and content were determined by the learning outcomes – this would be typical of EBL which is strongly process driven. In addition, the EBL modules have learning outcomes that relate to different levels and components: module outcomes, but then more detailed and specific outcomes relating to individual assignments of problems. This, in our view, was essential for guiding both teachers and students through a learning style that is necessarily less structured than the traditional mode of delivery.

2. What were your main reference points (internal and external)?

Prompts: Did you consult Tuning Group descriptors, UK subject-benchmark statements, documentation from professional institutes, NQAI documentation? 

Was there a consensus about what are the most important things that students of your discipline should learn in the selected programme (graduate attributes)?

We did look at some of the UK subject statements, and some of the NQAI documents (this latter mostly in relation to guidance about what was appropriate to each level), but beyond that we tried quite hard to articulate our own vision which we then expressed in language appropriate to the task, using, amongst other things, guidelines internal to UCD. There was a pretty clear consensus about attributes, but less agreement on the means by which these should be achieved.

3. What challenges did you encounter in the drafting stages and how did you overcome them?

There was initial resistance to this kind of approach, so it was incumbent upon us to demonstrate how learning outcomes can actually streamline and focus teaching, by providing structure and examples. There’s an important segment around application and implementation, namely how the outcomes can be properly embedded in the curriculum and in assessments. This requires training and guidance that is difficult to put in place given resources and pressures on time – only then do learning outcomes become anything more than aspirational statements that threaten to float away from actual practice.

4. What have been the benefits (if any) of drafting and working with learning outcomes at programme level and at module level?

The development of clarity and coherence across the programme, and a more logical sense of how the student progresses as they move through the programme. The process of articulating what students should be able to do was very useful and increased staff confidence and interest in their teaching.

5. What has been the impact of the programme learning outcomes on teaching, assessment, and on student behaviour and performance?

They have enabled us to be far more consistent about delivery and assessment, as the learning outcomes have become the benchmark by which individual assignments are judged. This has been particularly useful in judging group work, where individual students have sometimes questioned their grade. In terms of student behaviour learning outcomes have not yet had as much of an impact as we would like – first years in particular are so content-driven that they find it hard to extrapolate to skills and competencies. In future years in EBL at least, we plan to focus more explicitly on learning outcomes and encourage students to use them to assess their own progress. We have seen an improvement in student performance, particularly in the B and C grade bands, and a reduction in the fail rate – this could be due to many factors (the EBL method, the impact of group work etc), but owes something at least to the clear and consistent grading enabled by coherent learning outcomes.

6. Free comments

I was initially something of a sceptic about learning outcomes, believing (like most academics) that what I was trying to do must be self-evident. But the process of sifting through and being selective about what should achieved in a given context has been most useful and has significantly improved course design and delivery as well as strengthening student learning.

Discipline

Music

Programme

Bachelor in Music Education (Concurrent Honours Bachelor degree and second-level teaching qualification, NFQ Level 8)

ECTS credits 240+ over 4 years.

College

Trinity College Dublin in association with the Dublin Institute of Technology Conservatory of Music and Drama and the Royal Irish Academy of Music.

1. Please outline the process you adopted in order to arrive at an agreed set of programme outcomes. 

Note: By ‘process’ is meant both the local drafting process, and the verification process at institutional level.

After some internal discussion within TCD on the best way to approach the process of devising learning outcomes, the Course Co-ordinator for the Music education degree devised the outcomes. Shared initially with the School of Education’s Director of Teaching and Learning (Undergraduate), the draft outcomes underwent further development in terms of content, format, expression and clarity. During this iterative process the number of learning outcomes was increased. Subsequently, the draft outcomes were circulated to key personnel involved in the planning, administration and running of the B.Mus.Ed. programme. Constituted as the Course Co-ordinating Committee, this group includes representatives of the Schools in Trinity which contribute to the programme (namely Education, Music, and Histories and Humanities) and of the partner institutions, namely the Royal Irish Academy of Music and Dublin Institute of Technology Conservatory of Music and Drama. The learning outcomes were circulated in advance of the meeting and thoroughly discussed by representatives. The programme outcomes were then forwarded to Trinity’s learning outcomes project ‘triage group’ for comment and feedback.

2. What were your main reference points (internal and external)?

Prompts: Did you consult Tuning Group descriptors, UK subject-benchmark statements, documentation from professional institutes, NQAI documentation? 

Was there a consensus about what are the most important things that students of your discipline should learn in the selected programme (graduate attributes)?

Trinity College offered a number of working seminars on drafting learning outcomes, facilitated by the Bologna Desk of the Vice-Provost’s Office. These seminars proved useful in raising awareness and providing possible strategies. More focused local meetings helped focus attention on devising learning outcomes in general, primarily using the existing music education programme objectives and content and guidelines. As part of the process of conceptualizing the outcomes, consideration was given to visualizing the qualities required in potential post-primary music teachers.

Other reference points included a music education conference in Exeter attended by the B.Mus.Ed. Course Co-ordinator in which the ‘Tuning’ Music Working group presented their initial report on learning outcomes. The preamble to, and philosophy behind, their attempt to write comprehensive learning outcomes for music education was very informative. They also attempted to define learning outcomes and put their significance in the context of music education, and were on hand to answer questions from the floor. This informal contact was very useful.   Other reference points emerged from the development of programme learning outcomes for the other main TCD School of Education undergraduate programme, the Bachelor in Education. This was underway at the same time as drafting for the B.Mus.Ed. and dialogue between those responsible for each programme was mutually beneficial.

3. What challenges did you encounter in the drafting stages and how did you overcome them?

The initial learning outcomes numbered about seven. To facilitate greater clarity and in light of the complex multi-faceted nature of the programme and its delivery, some dis-aggregation of learning outcomes was undertaken to reflect more fully the breadth and depth of the student learning experience within and outside Trinity. Whereas the number of outcomes probably ultimately exceeded what was initially envisaged, the broad range of programme elements necessitated similar breadth in learning outcomes.   Getting to grips with the specific language of learning outcomes promoted in the Bologna initiative - the ‘normal’ way to write them up- was another challenge, though perhaps less so than in other Schools given the familiarity of School of Education staff with the nomenclature and concepts of aims, objectives and outcomes. In addition to this it was a challenge to sustain enthusiasm for a task that seemed at times to be very bureaucratic. This lessened as the process gathered momentum, but it was a factor in the early stages.    Understandably, given the breadth and scale of the programme, along with their other professional commitments, participating staff are extremely busy and securing their time to focus on the learning outcomes initiative was not always straightforward. As the process matured and especially once draft outcomes were available for discussion, securing the involvement of most decision-makers was more readily achieved. 

4. What have been the benefits (if any) of drafting and working with learning outcomes at programme level and at module level?

There have been benefits in terms of greater insight and understanding of the B.Mus.Ed. programme. Devising learning outcomes provided another opportunity for reflecting on the programme - of asking ourselves “is what we are teaching central to what a music teacher needs to know?” It has encouraged us to look to the future of music education, and has reminded us that our programme needs to anticipate this future and offer student the relevant skills ands knowledge for this time. It has given us the opportunity to weigh the relative importance of each skill as a facet of the overall programme.

It also helps to clarify our vision of what we believe a great music teacher to be, and to strive to facilitate development to this level for all of our students.

For new staff becoming involved, the availability of learning outcomes offers an efficient, effective way to get to know the programme, starting with the important vision and philosophy as articulated in the programme learning outcomes.

5. What has been the impact of the programme learning outcomes on teaching, assessment, and on student behaviour and performance?

Our learning outcomes have only recently been developed so we have experienced impacts from the drafting process only. The impact we hope to achieve includes clarity for employers in what they can expect from a music education graduate of this programme; clarity for our students as to what to expect from the programme, and clarity for lecturers as to what their targets are in teaching the students.

6. Free comments

The process of devising learning outcomes at programme level has assisted in providing clarity and transparency for our degree. Beginning with devising learning outcomes at programme level has given us the destination and the overall vision of what we want to achieve. The time and energy required to bring such a process to fruition ought not be under-estimated, especially where staff from different Schools and Colleges are involved in planning and teaching the same programme. Such an initiative is worthwhile, but it requires considerable time and commitment on the part of academic and administrative staff.

Discipline

Physics

Programme

B.Sc. in Applied Physics/Physics and Astronomy, NFQ Level 8

College

Dublin City University

1.  Please outline the process you adopted in order to arrive at an agreed set of programme outcomes Note: By ‘process’ is meant both the local drafting process, and the verification process at institutional level.

The teaching convenor and programme chairs looked at the existing degree accreditation documents, the Tuning and IOP (Institute of Physics) documents and made a first draft. This was then circulated to all the staff teaching on the programmes and revised in light of their comments.

Using the template provided within the University the physics programme outcomes were then looked at internally in DCU by the AFI* and Teaching and Learning staff, who made some minor changes and they were then submitted for external evaluation.

Further amendments were made on foot of comments from the external Validation Panel.

 * AFI - Academic Framework for Innovation; a curriculum reform project in DCU within which the change to Learning Outcomes is being carried out.

2. What were your main reference points (internal and external)?

Prompts: Did you consult Tuning Group descriptors, UK subject-benchmark statements, documentation from professional institutes, NQAI documentation? 

Was there a consensus about what are the most important things that students of your discipline should learn in the selected programme (graduate attributes)?

Sources:

• Internal degree accreditation documents (especially the more recent Physics/Astronomy one).
• Tuning Project: Reference points for the design and delivery of Degree Programmes in Physics.
• Institute of Physics document :The Physics Degree (Core of Physics).
  http://www.iop.org/activity/policy/Degree_Accreditation/file_26578.pdf
• DCU Award Learning Outcome template.

There is a wide consensus across Europe about the contents of a Bachelor Physics degree, which is reflected in the Tuning document. However, there is a difference in approach between continental Europe and the more experimental Anglo-American tradition. Because of this the IOP document proved to be a better guide as it is more detailed and corresponds closely with the approach adopted in Irish Universities. 

3.  What challenges did you encounter in the drafting stages and how did you overcome them?

The main difficulty was in knowing where to begin as most staff were unused to the concept of learning outcomes. Once the initial difficulties were overcome the drafting was reasonably straightforward. Some difficulties were experienced in getting the wording right and some staff felt that there was a too rigid approach to using the “correct” words in the final version.

4. What have been the benefits (if any) of drafting and working with learning outcomes at programme level and at module level?

At programme level there is a benefit in looking again at the degree content and how it fits together. This is especially important in the Applied Physics programme which was designed 25 years ago and has undergone many piecemeal changes since then.

The module learning outcomes are being drawn up at present. This should allow a focus on the topics really required and later on how the various modules correlate with the overall programme structure. This will be done in the near future.

5. What has been the impact of the programme learning outcomes on teaching, assessment, and on student behaviour and performance?

None to date as the process is not yet compete.

6.  Free comments

Experience has shown that staff are much more comfortable with this process if they are presented with an appropriate template for both programme and module learning outcomes. It is also vital that the appropriate supporting software is in place and working before the process starts. Delays in providing the appropriate backup can lead to extremely short deadline for the staff producing the learning outcomes.