 | Too long?
I suspect this text to be too long. We do not know till I get the booklet template I've asked EUN to send me! |
| Slightly shorter?
I have reduced the length by about 100 words. Would it make more impact if the recommendations came after the introductory remarks? |
Assessment Interoperability
Assessment interoperability concerns are focused on computer based testing for summative and formative purposes.
At present electronic assessment is most likely to be encountered as a set of discrete tasks within the context of an adopted Learning Management System LMS of Virtual Learning Environment VLE, and mechanisms have been devised that allow basic interoperability at a granular level of well established and used types of question. However, the market is prompting new models for the delivery of assessments. These are based on integrated Web services provided by a variety of organisations rather than vertically integrated offerings from a single vendor.
Several barriers have impeded the drive towards assessment interoperability; the institutional tendency to adopt a specific VLE, the limited availability of central assessment resource of perceived value, vendor interest in locking in customers to their product, and the growth of Web delivered assessment services. LIFE experts fear that, without intervention, the drivers for interoperability in future marketplaces may well remain ineffective.
Assessment specifications state of art
The de-facto standard for assessment interoperability is provided by the IMS Question and Test Interoperability (QTI) specification. This describes a model for the representation of questions, tests and the reporting of their results.
There has been an ambiguous level of acceptance of QTI by the developer community, and, to date, interoperability has not been as high a priority as public statements suggest. The market evidence is that customers are most comfortable with monolithic systems that offer the end-user integrated functionality and that developers respond to this by aspiring to but not prioritising interoperability.
Relevant standards in assessment interoperability include the following.
- Assessment creation: IMS QTI
- Assessment task definition: IMS Reusable Definition of Competency or Educational Objective (RDCEO).
- Assessment Discovery: IEEE Learning Object Metadata Standard LOM for describing the items;
- Assessment ordering: The HR-XML Assessments and Assessment Catalog specifications support ordering assessments in a standard way.
- Assessment distribution: The IMS Content Packaging specification is designed to support the electronic distribution of learning and training materials.
- Assessment reporting and status: IMS QTI, the Aviation Industry Computer based training Committee AICC standard, the Shareable Content Object Reference Model SCORM, the Human Resource Extensible Mark-up Language HR-XML, and the Schools Interoperability Framework SIF (from the K-12 community).
Issues
The present discussion of interoperability is limited mainly to automated computer-mediated testing. However, assessment that is authentic will be based around a much broader range of diverse tasks. These include; the production of specified documents (e.g. essays, reports etc.), engagement in real or simulated professional roles (e.g. laboratories, fieldwork, training roles in organisation, virtual practice environments, etc.), public performance (e.g. in giving presentations, playing music or sport, etc.), self reflection (e.g. in portfolio construction), and reporting on prior experience. All pose interoperability issues.
The intimate relationship between the summative and formative functions of assessment is evident in the tendency for students to adopt a pattern of study that it is geared towards meeting the summative requirements while welcoming teacher advice and feedback. It follows that assessment should be designed so that it links to support functions. Mechanisms for interoperability should support this integration of assessment within learning.
There is no definition of compliance with assessment interoperability standards and no formal accreditation procedures, allowing products to claim standards compliance without any true interoperability.
To realise a scenario that involves broad usage of items and or tests, work will be required to establish quality assured procedures for item production, to maintain a valid bank of items for test construction with the appropriate statistics, and to govern item exposure.
An analysis of a number of interoperability scenarios, though incomplete, suggest that the most promising area in which interoperable assessment systems might demonstrate value is formative assessment for all educational sectors but most particularly, given the size of the potential market, the school sector.
Recommendations
1. In order to promote adoption of interoperable assessment practice and demonstrate that a self-sustaining market for interoperable resources and systems can exist, projects should be undertaken that will yield content of demonstrable value across institutional and national boundaries. These projects should be focused on formative assessment resources for all educational sectors with priority given to the largest markets.
2. A high priority should be accorded to the provision of open source assessment related tools for developers and users. Consideration should be given by IMS to the construction of a QTI 'Lite' Version 2 which is less complex, and easier to understand and implement. The existence of such a basic standard would facilitate interoperability at lower cost.
More information
