|Teaching and Learning Forum 2010 [ Refereed papers ]|
Jianhong (Cecilia) Xia and Shelley Yeo
Faculty of Science and Engineering
Curtin University of Technology
Managing the diversity of student backgrounds in today's complex higher education classrooms is a challenging task for lecturers. This paper reports on action research involving a complex nested group of units in Geographic Information Science (GIS) delivered to a diverse student body, including international and domestic students from different disciplines studying in on-campus, online and distance education modes. GIS is a technology-dependent discipline of relevance to a wide variety of other disciplines including, for example land management, urban development or health sciences, to facilitate land-use planning, map disease distribution, analyse spatial patterns of crime or model urban growth. The concern of the lecturer was that adaptation of the learning activities necessary for accommodating flexible modes of delivery and student backgrounds would impact on student assessment outcomes. The research findings indicate that while there were some significant differences in assessment outcomes associated with student backgrounds and discipline, no differences could be ascribed to, for example, language background of students. This paper discusses the study and seeks explanations for the results. The implications for teaching and further action research with this unit in order to sustain student numbers and teaching and learning standards in the future are also presented.
This paper reports on an action research project undertaken by an early career lecturer from a non-English speaking background (NESB) who is teaching a group of units which incorporates all of the above complexities. The research focuses on determining the impact of diverse student backgrounds and flexible delivery modes on assessment outcomes with the aim of being fair to all students. Lessons and insights will be derived from this study and future action will be designed in order to sustain the number of students and standard of teaching and learning in this class.
Universities delivering programs in transnational settings are also subject to consensual or legislative requirements in relation to quality control and quality assurance assessment processes (AVCC, 2005). Higher education institutions in Australia are subject to the National Protocols for Higher Education Approval Processes (MCEETYA, 2000) that refer to the expectation of "equivalent" standards for Australian universities operating offshore. While the notion of equivalence is not fully understood, it can be taken that students' experiences, if not identical, should afford them the same opportunities to learn and to demonstrate their knowledge as students in the home university (DEST, 2005).
Gipps (1995) uses the term equity which is interpreted as the spirit of justice: "The concept of equity in assessment as we use it implies that assessment practice and interpretation of results are fair and just for all groups" (p. 273). Gipps' focus on equity in relation to assessment includes both the definition as well as the practices of assessment. Assessment opportunities should be equally available to all groups and presented in such a way that all groups feel able to participate fully. This, however, does not imply equality of outcome because other factors, for example student motivation and esteem, and teacher behaviour and expectations, may impact on student achievement. On the other hand, two of the key steps involved in ensuring fair assessment are the appropriate interpretation of students' results (in relation to the tasks and assessment standards) and the evaluation of assessment outcomes (in relation to student and teacher expectations) (Suskie, 2002).
GIS technology has been applied extensively to a variety disciplines such as land management, urban development, transportation management and health sciences to facilitate land-use planning, analyse spatial patterns of crime or model urban growth or map the distribution of disease. Students enrol in GIS units to learn about the structure, function and use of geographic information systems, the input, management, analysis and presentation of geographic data and relationships, the storage and manipulation of vector and raster data, digital terrain modelling, the integration and management of spatial and attribute data, including applications and examples. Students must be competent in computing and mathematics.
In this situation, the problem under investigation is the equivalence of assessment outcomes for different sub-groups of students in the GIS nested units GIS 181, GIS 581 and GIS212 which have multiple delivery modes and locations, and adaptations of assessment that are necessary to accommodate the different delivery modes. The outcomes of this aspect of the research will be used to plan for on-going improvement in the units.
We developed four null hypotheses (H0) to test student differences on assessment results in four dimensions: location of study, student language background, student maturity and discipline of study. Each dimension represents a potential barrier to equality of learning and assessment.
|Mode of study||H0||There is no difference between the assessment results of on-campus and distance education students.|
|H0||There is no difference between the assessment results of domestic and international students.|
|Student maturity||H0||There is no difference between the assessment results of undergraduate and postgraduate students.|
|Discipline of study||H0||There is no difference between the assessment results of students from different disciplines (courses).|
The statistical comparison method adopted were unpaired t-tests to compare two groups and one-way ANOVA with post hoc analysis to compare three unmatched groups. The null hypotheses will be rejected if the p-value is less than 0.05.
Table 1 summarises the mean scores for different subgroups of students on each of the assessment items. Tables 2 and 3 present statistical data showing any significant differences that were found.
|Sub-groups||Mean assessment results|
|N||Mean||St. Dev||t||Sig (2-tailed)|
|F||Sig.||GIS - Survey||GIS - Geology|
The distance education students are supported with a web-based teaching tool. All the lectures are screen captured and synchronised with auto file. The students were also sent weekly emails which addressed important issues related to lectures and tutorials, and were encouraged to send emails to the lecturer if they encountered any problems regarding the online lectures and tutorials. It was apparent during the semester that distance students were more active in using the discussion forum in WebCT to communicate with and help each other. It is possible that this community-based teaching and learning tool served to encourage and motivate the distance education students to develop their self-learning skills.
While on-campus students can obtain direct help from tutors and other students when they have problems, distance education students are relatively more isolated. When they have faced difficulties in tutorials or assignments, distance students have had to be more proactive in seeking help to resolve their problems. Although they may have, at times, they faced a lot of frustration, they also appear to have developed better problem-solving skills than on-campus students.
We can only speculate on the reasons for these differences, particularly as we do not know if there were other differences between these groups which might account for the results. Students studying GIS or Surveying may be more self-motivated because the unit content bears a closer relationship to the core business of their course, compared with the Geology students. For Geology students, the unit is offered by a Department other than their own. In addition, there is a higher requirement for mathematics and computing skills which may not be as well emphasised or needed in the Geology program.
This paper reflects the concerns of some academics who teach today's complex higher education classrooms. The academic who is the subject of this paper simultaneously teaches in flexible mode (online and face-to-face) to students who are both undergraduate and postgraduate, from both English and non English speaking backgrounds and who are in three different degree programs. A review of these students' assessment results has been used to examine whether or not there may be unfairness associated with flexible delivery and student diversity. The adaptation of delivery method and assessment for the offshore, distance education students appear to have had little effect on the students' results, thus giving some confidence that these methods are at least equivalent as experienced by the students. There is some evidence that NESB students, despite their struggles with language, have performed at least as well as their English-speaking counterparts, although this is not to say that they have not been disadvantaged by having to learn and perform assessment in English. The most pervasive difference is in the results of students enrolled in different degree programs, and we attribute this more to the motivation and past or concurrent experiences of the students concerned rather than differences in teaching or assessment methods or materials resulting from the imperative for flexible delivery. The current work practices and perhaps maturity of the postgraduate students may explain the apparent advantage that they have over the undergraduate students. Thus, we are of the view that for this cohort of students there is little evidence of unfairness associated with delivery or assessment methods. However, our observations have led us to consider future improvements.
Braddock, R., & Loxton, J. (2003). Impacts of internationalisation on teaching and learning. Paper presented at the The Learning Conference, Institute of Education, University of London, 15-18 July, 2003. http://lc03.commongroundconferences.com/ProposalSystem/Presentations/P000823.html
Chang, K. (2009). Introduction to Geographic Information Systems (5th ed.). Sydney: McGraw-Hill.
DEST (2005). A National Quality Strategy for Australian Transnational Education and Training: A Discussion Paper. Canberra: Commonwealth of Australia. http://pandora.nla.gov.au/pan/53520/20051021-0000/aei.dest.gov.au/AEI/GovernmentActivities/QAAustralianEducationAndTrainingSystem/QualStrat_pdf.pdf
Dick, B. (1997). Action learning and action research [Electronic Version]. Retrieved 13/10/09 from http://www.scu.edu.au/schools/gcm/ar/arp/actlearn.html
Dunn, L., Morgan, C., O'Reilly, M., & Parry, S. (2004). The student assessment handbook. London: Routledge Falmer.
Elton, L., & Johnston, B. (2002). Assessment in Universities: A critical review of research. [verified 24 Jan 2010] http://eprints.soton.ac.uk/59244/01/59244.pdf
Farkas-Teekens, H. (1997). A profile of the ideal lecturer for the international classroom. In H. Farkas-Teekens & M. van der Wende (Eds.), Teaching in the International Classroom. Nuffic papers 8. Amsterdam: Nuffic.
Gipps, C. (1995). What do we mean by equity in relation to assessment? Assessment in Education: Principles, Policy & Practice, 2(3), 271-281.
Goodfellow, R., Lea, M., Gonzalez, F., & Mason, R. (2001). Opportunity and e-quality: Intercultural and linguistic issues in global online learning. Distance Education, 22(1), 65-84.
James, R., McInnis, C., & Devlin, M. (2002). Assessing learning in Australian universities: Ideas, strategies and resources for quality in student assessment. Melbourne: Centre for the Study of Higher Education. http://www.cshe.unimelb.edu.au/assessinglearning/
Kember, D., & Kelly, M. (1993). Improving teaching through action research. Campbelltown, NSW: Higher Education Research and Development Society of Australasia.
Ling, P., Arger, G., Helen Smallwood, H., Ron Toomey, R., Denise Kirkpatrick, D., & Ian Barnard, I. (2001). The effectiveness of models of flexible provision of higher education. Canberra: DETYA.
MCEETYA (2000). National protocols for higher education approval processes. Canberra: Ministerial Council on Education, Employment, Training and Youth Affairs. http://www.mceetya.edu.au/mceecdya/national_protocols_for_higher_education_mainpage,15212.html
O'Brien, R. (1998). An overview of the methodological approach of action research. Retrieved 29/9/09, from http://www.web.ca/~robrien/papers/arfinal.html
Pyvis, D., & Chapman, A. (2004). Student experiences of offshore higher education: Issues for quality. AUQA Occasional Publications Number 3. Melbourne: Australian Universities Quality Agency. http://www.auqa.edu.au/files/publications/pyvis_chapman_report.pdf
Scarino, A., Crichton, J., & Papademetre, L. (2006). A framework for quality assurance in the development and delivery of offshore programs in languages other than English. London: The Observatory on Borderless Higher Education.
Suskie, L. (2002). Fair assessment practices: Giving students equitable opportunities to demonstrate learning [Electronic Version]. Adventures in Assessment, 14 from http://sabes.org/resources/publications/adventures/vol14/14suskie.htm
|Authors: Jianhong (Cecilia) Xia and Shelley Yeo, Faculty of Science and Engineering, Curtin University of Technology. |
Email: firstname.lastname@example.org, email@example.com
Please cite as: Xia, J. & Yeo, S. (2010). The impact of diverse student backgrounds and flexible delivery modes on assessment outcomes. In Educating for sustainability. Proceedings of the 19th Annual Teaching Learning Forum, 28-29 January 2010. Perth: Edith Cowan University. http://otl.curtin.edu.au/tlf/tlf2010/refereed/xia.html
Copyright 2010 Jianhong (Cecilia) Xia and Shelley Yeo. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.