Category: Professional practice
|Teaching and Learning Forum 2010 [ Refereed papers ]|
The University of Western Australia
Sustainability in higher education can be defined in the approach we take to teaching our students - by using sustainable practices in our definitions of good teaching, and incorporating them in to the expectations we have for teaching staff, we can create a sustainable teaching and learning environment. In 2008, as part of UWA's involvement in the national ALTC Teaching Quality Indicators (TQI) project a new framework of teaching criteria was developed to better support good quality teaching and learning within the institution, and enhance the value placed on teaching by enabling more robust developmental and evaluative systems for promoting, recognising and rewarding good teaching.
The framework is premised on a shared understanding of good teaching which has been developed through extensive consultation across the University throughout the project, and which is based in the research evidence about the types of teaching practice which improve student learning outcomes. The project has developed this shared definition of good teaching, along with providing a structure around which to talk about teaching, and the associated development of examples and guidelines of sources and types of evidence staff can collect which can demonstrate the achievement of good teaching. The new teaching criteria framework underpins the teaching component of the academic portfolio, and will therefore inform all of the University's reward and recognition processes from developmental annual review discussions, through to promotion applications. By robustly defining what is meant by good teaching, and assisting staff to understand how to demonstrate it, the framework may be able to improve the value placed on teaching as an academic endeavour and go some way to supporting sustainable education.
This paper describes the process UWA went through in developing the new framework, the way that good teaching has been defined in the UWA context, and how the resulting framework is being used to enhance and support good teaching practice.
In 2008 the University of Western Australia (UWA), developed a new framework of teaching criteria designed to underpin the teaching component of the academic portfolio and provide a robust, clear, and consistent definition of good teaching which was appropriate to the UWA context and grounded in the research. The resulting teaching criteria framework aims to provide guidance to both academic teaching staff and their supervisors in evaluating teaching quality - providing a consistent language with which to talk about the teaching that we do, clarifying the types and sources of required evidence and a framework within which both developmental and summative discussions can be held.
This paper commences with a brief background to the project, before discussing the development of a University-wide definition of good teaching, followed by a description of the process of developing the teaching criteria framework to support that definition, the mechanisms which can be used to demonstrate good teaching, and the way in which the framework will be used at UWA to inform the reward and recognition of teaching and teachers.
The TQI framework consists of four dimensions which have been identified as affecting teaching quality:
As a pilot institution, UWA chose to focus its efforts for implementing the framework in the dimension of 'Institutional Climate and Systems', and in particular on the promotion, recognition and reward of teaching. The research indicates that exposure of students to good practices in undergraduate education is positively related to gains in student learning (Kuh, Pace & Vesper, 1997); that by encouraging good teaching, good learning results. The evidence also shows that although traditionally teaching quality has been left up to the individual academic teacher, and good teaching considered to be a personal trait that cannot be taught (Keig & Waggoner, 1994); in fact the institutional culture of a university, and the way that teaching and learning is valued by that university and systematically promoted, recognised and rewarded does have an impact on the quality of the teaching which results (Chalmers, 2007). Good teaching is not mysterious, there is a large body of research about good teaching practices in higher education, and it is possible to measure the quality of teaching that our staff are achieving - for example, evidence of a student-centred learning approach is a strong indicator of teaching quality (Chalmers, 2007).
Recognising a lack of clarity in existing evaluative systems for teaching and learning at the individual level, and alongside other projects aimed at developing institutional level indicators, the University set out to develop a new set of teaching criteria to enable evaluation of teaching at the individual level, for developmental and summative purposes, which could apply to all staff regardless of discipline and be used consistently across the University.
A review of the literature reveals broad consensus over a significant period of time about the types of teaching practices which are more likely to lead to learning, and the ways in which they can be evaluated. From the Seven Principles of Good Teaching (Chickering & Gamson, 1987) in the late eighties, through Boyer's introduction of the concept of 'scholarship' in teaching (Boyer, 1990) and expressed in Australia through the ALTC Teaching Award criteria, the same principles arise again and again. Although the "list" of good teaching practices is long, and necessarily diverse, it can be summarised as follows, noting that the references provided here are examples only of a large body of research. Good (sustainable) teaching is student centred and culturally inclusive (Chalmers, 2007), it engages students with the subject matter through active learning (Braxton, 2006); provides prompt and meaningful feedback (Chickering & Gamson, 1987) provides students with learning outcomes that are aligned with assessment (Biggs, 2003) and is enhanced when the teacher is engaged with the scholarship of teaching and learning (Boyer, 1990).
The research undertaken for this project identified four main instruments available for evaluating the quality of teaching - student evaluations (Barrie et al 2007); student learning (Chalmers, 2007); peer review; and self-reflection (Atwood, 2000). All four are valid and important ways of measuring the success of an individual's teaching and bringing about improvement; each may be suited to the evaluation of different aspects of teaching practice, and has its own strengths and weaknesses. It is therefore important not to rely on a single evaluation tool, but to use a number of instruments to measure quality (Barrie et al, 2007) and ensure that a wide range of practices are evaluated, from classroom practice to teaching and learning materials, and student learning outcomes. This recognition that a robust evaluation requires evidence not only from a number of different sources, but also different types of sources, is one of the main principles arising out of the TQI project, which attempts to strike a balance between qualitative and quantitative measures of quality (Chalmers, 2007).
Once good teaching had been defined through the review of the literature, an initial consultation period was undertaken with the University community to clarify the scope of the project, and set out a series of principles upon which any evaluative system for teaching and learning should be based. This enabled the project to be placed specifically in the UWA context, and gave stakeholders the opportunity to contribute not only to a robust definition of quality, but also to the approach the University should take in its efforts to better promote, recognise and reward good teaching. A number of principles were set out for any evaluative system, which emphasised that the quality of teaching was an institutional responsibility, not just an individual one; that evaluation of quality should be systematic, robust and comparative; criteria used for teaching needed to be consistent across all mechanisms, and based in the current research; that criteria need to be contextualised by discipline, detailed but not prescriptive, differentiated by level, and enable developmental approaches to enhancement. At this stage, a number of desired outcomes for the project were also articulated - that robust teaching criteria and evaluative systems would raise the status of teaching in the institution; improve teaching quality; and enable the identification of strengths and weaknesses, and targeted professional development.
Following this consultation period further research was undertaken to identify a number of alternative models for an evaluative system at UWA. A desk-top audit of practices at other Australian institutions was undertaken, along with the identification of innovations internationally. A number of approaches in particular to criteria used for promotion processes were identified within the Australian context, and from this it was proposed that a model was required which combined an aligned structure across the University (consistent definitions of quality for all purposes) with the use of standards to differentiate between career levels. Different measures (qualitative and quantitative) could be used as evidence to demonstrate how each of the qualitative criteria were being met, to the level set out in the relevant standard. This schema is supported by the research which suggests that evaluative systems which use a qualitative approach incorporating a developmental cycle are most effective in driving improvement (Garlick & Pryor, 2004).
In identifying the need to implement an evaluative system which provided a consistent set of expectations for all purposes it was proposed to develop a new framework for staff preparing the teaching component of the academic portfolio. By concentrating on reform at this level, it would be possible to devise a set of University wide expectations and definitions of quality which would permeate all of the reward and recognition processes associated with teaching - from developmental discussions with supervisors and identification of professional development opportunities through to formal annual review processes, probation and tenure requirements and promotion applications, the same underlying principles, expectations and definitions of quality could be used consistently across the University and would translate externally.
The research undertaken for this stage of the project had identified the UK Higher Education Academy's (HEA) Professional Standards Framework as one whose underlying concepts, and also its practical definitions of good teaching quality, aligned well with the University's perspective and focus. Rather than developing a whole new system, it was therefore proposed to use the HEA framework as the basis for developing a new set of criteria for UWA.
The UK HEA Professional Standards Framework was developed as a way of accrediting professional development programmes for teaching in the UK and also as part of the professional recognition scheme which confers fellow status on individual higher education teachers. There are three standard descriptors each of which is applicable to a number of staff roles and to different career stages of those engaged in teaching and supporting learning. The standard descriptors are underpinned by areas of professional activity, core knowledge and professional values. The framework provides a reference point for institutions and individuals as well as supporting ongoing development within any one standard descriptor. (HEA, UK website, 2009)
A further extensive consultation period was undertaken involving academic leaders at the University (Deans, Heads of School, Associate Deans (Teaching and Learning) through which the Professional Standards Framework was re-worked for the UWA and Australian context, and a version of the framework developed to form the basis for the teaching component of the academic portfolio to underpin all evaluative processes relating to teaching at UWA. This consultation period was invaluable in determining how to structure the framework, how to guide its use and what emphasis was appropriate in the UWA context in relation to teaching expectations for academic staff. This consultative period revealed the extent of discipline variation in expectations for teaching staff, and led to the very flexible nature of the resulting framework.
More substantial evidence is expected of staff as they move through the career levels, and for each area of activity examples of the types and sources of evidence which may be appropriate are provided to assist staff in developing their portfolios.
To be useful in a practical sense, the framework would need to be able to be contextualised by disciplines across the University. The types of activities staff are engaged in at different levels of their career varies significantly across different disciplines, which was revealed when consultation took place on the examples of sources / types of evidence for each area of activity. Faculties and Schools of the University have been invited to contextualise the framework to their own needs - to emphasise some activities over others, or to define which types of evidence would be expected at different levels of an academics' career. This flexibility within the framework means that it can be used to develop standards of teaching that are meaningful to the individual academic teaching bodies, whilst retaining a consistent definition of what the University expects of its teachers. All schools and faculties of the University are engaged in a process of defining standards in their discipline contexts, not just for teaching and learning but for all aspects of the University's role (teaching, research, service) and the teaching criteria framework is now informing that process for teaching by providing schools and faculties with a consistent structure within which to define discipline specific standards, a common language to talk about expectations, and examples of the kinds of measures which can be used not just at the individual academic level, but also for departmental level evaluations.
In addition to the framework itself, standard descriptors for each academic career level set out general expectations for the teaching component of an academic staff member's role. These descriptors support the Minimum Standards for Academic Levels (MSAL) by providing more specific quality expectations (rather than defining types of activities), and also support the UWA Promotion criteria by providing guidance on the way that the promotion criteria relating to 'high quality teaching' can be demonstrated at an appropriate level. The standard descriptors retain the UK Professional Standards Frameworks' emphasis on the student learning experience, and the teaching-research nexus, as these relate directly to the UWA mission and objectives. Each teaching standard describes expectations in terms of teaching quality, commitment to continuous development, and leadership (UWA TCF website).
There are some aspects of teaching that cannot be evaluated effectively by students, for example student evaluations cannot provide an assessment of elements such as the teaching content, pedagogical content and ethical standards of practice - these dimensions of quality teaching are best assessed by colleagues (Keig & Waggoner, 1994) and this is where the use of peer review in evaluating teaching quality is most appropriate. In this context peer review is interpreted broadly and includes the review of learning materials; observations of teaching; mentoring arrangements and teaching communities (Atwood et al, 2000). In addition, the TQI framework identifies evidence like testimonials from colleagues on the same curriculum team or committee as further legitimate peer review. The recognition that peer review is an important aspect of teacher evaluations has been gradual, however following the introduction of the new Teaching Criteria Framework, the University has now also formalised a set of Guidelines for Peer Review in the form of a good practice guide after a number of years of discussion and development. Whilst the University does not require staff to use peer review as a teaching evaluation method, it does strongly encourage its use.
Self-evaluation, incorporating the principle of self-reflective practice, remains an important aspect of evaluation, and is crucial in the process of interpreting the other evidence gathered in to a coherent analysis of a staff member's teaching practice within the framework structure.
Evidence of student learning is perhaps the least-utilised measure of teaching quality at present. Such evidence includes external moderation of assessment tasks, external marking / moderation of student work (including of the quality of student attainment of understanding of key discipline concepts), accepted tests of student learning (e.g. pre- and post-testing), or official university statistics (e.g. for research student completions). By giving specific examples of the types of evidence that may be appropriate when demonstrating particular aspects of each teaching activity, it is hoped that the framework will assist staff in using a wider range of evidence to evaluate their teaching and in particular encourage the use of direct evidence of learning.
Because the framework was developed through an extensive consultation process, feedback from staff has already been incorporated in to the final framework, and most of the issues raised have been addressed. For example, the framework incorporates innovation as a core principle for quality teaching, following concern that using a categorising framework as an evaluative tool had the potential to limit innovative teaching practices. Concerns around workload issues remain, however through comparison with current requirements for teaching portfolios and other evaluative tasks such concerns can be minimised. The framework has its limitations, it can look daunting and complex, but may in fact over simplify expectations around teaching, and this a reflection of an attempt to provide a University-wide framework whilst also providing guidelines and examples that staff can use. The framework will be monitored over time to evaluate its impact on enhancing the sustainability of the student learning experience offered at UWA.
Implementation of the new criteria is staggered, with staff only interacting with the new framework when they come to revise or create their academic portfolio, and staff will not be expected to have a portfolio which aligns to the new framework for a number of years. Over time more staff will go through the process of engaging with the framework, collecting different types of evidence to support their claims for good quality teaching, engaging in professional development around developing a portfolio, and discussing the standard descriptors with their supervisors. Similarly, schools and faculties will go through a process of contextualising the framework for their discipline, and using it to assist in framing departmental level evaluations of teaching quality. As these activities occur across the University over the next couple of years it is hoped that the way in which teaching is discussed and considered within the University will evolve, and assist in the process of enhancing the value placed on teaching within the institution.
Whilst the project's longer term objectives of improving teaching quality remain to be seen, the process of defining good teaching in the UWA context and identifying robust tools for its evaluation is seen as worthwhile in and of itself. The ultimate goal is to develop a culture where good teaching is seen as an activity that is valued, respected, rewarded and recognised across the institution, and only by achieving this will we achieve the goal of sustainable education.
Barrie, Ginns & Symons (2008). "Student Surveys on Teaching and Learning: Final Report", Institute for Teaching and Learning, University of Sydney, for the Carrick Institute of Learning and Teaching in Higher Education, Retrieved 14 October 2009, from http://www.catl.uwa.edu.au/page/146823
Braxton, J.M. (2006). "Faculty Professional Choices in Teaching that Foster Student Success" National Postsecondary Education Cooperative, Retrieved 14 October 2009, from http://nces.ed.gov/npec/pdf/Braxton_report.pdf
Chalmers, D. (2007), "A review of Australian and international quality systems and indicators of learning and teaching" Carrick Institute for Learning and Teaching in Higher Education, Retrieved 13 October 2009, from http://www.catl.uwa.edu.au/__data/page/146827/T&L_Quality_Systems_and_Indicators.pdf
Garlick S. & Pryor G. (2004). Benchmarking the university: Learning about Improvement A report for the Department of Education, Science and Training, Retrieved 15 October 2009 from DEEWR Publication and Resources website: http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/benchmarking_the_university.htm
Higher Education Academy, UK (undated). Professional Standards Framework, accessed on 22 October 2009 http://www.heacademy.ac.uk/ourwork/policy/framework
Keig, L. & Waggoner, M.D. (1994). Collaborative peer review: The role of faculty in improving college teaching, ASHE-ERIC Higher Education Report No. 2. http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED378925
Kuh, G. D., Pace, R. C., & Vesper, N. (1997). The development of process indicators to estimate student gains associated with good practices in undergraduate education. Research in Higher Education, 38(4), 435-454.
The University of Western Australia, Teaching Criteria Framework website, accessed on 22 October 2009 http://www.catl.uwa.edu.au/tcf
|Author: Jacqueline Flowers, Centre for Advancement of Teaching and Learning, University of Western Australia. Email: Jacqueline.firstname.lastname@example.org
Please cite as: Flowers, J. (2010). Evidence of quality: The teaching criteria framework at UWA. In Educating for sustainability. Proceedings of the 19th Annual Teaching Learning Forum, 28-29 January 2010. Perth: Edith Cowan University. http://otl.curtin.edu.au/tlf/tlf2010/refereed/flowers.html
Copyright 2010 Jacqueline Flowers. The author assigns to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.