Quality Review of Assessments

The Learning Enhancement Directorate and all Faculties will have procedures in place for robust internal scrutiny of assessment designs and processes. Internal scrutiny takes several forms:

• Initial appraisal of assessments at the point of design via University evaluation/approval panels. (see planning and approval process). New course proposal forms (CA1) are signed off by Executive Dean/Associate Dean (AQSE) on behalf of the Faculty. Revalidation forms (CA6a) are signed off by the Programme/Subject Director, Associate Head of School and Associate Dean (AQSE).

• Programme/module revision via CA3 process. CA3 forms are approved and signed off by the Programme Director and the Associate Dean (AQSE).

• Centralised assessment review as a potential outcome of the annual CAQE process (Continuous Assurance of Quality Enhancement).

• Internal review of draft examination papers, marking schemes, coursework briefs and marking criteria prior to submission to the External Examiner to ensure assessment information aligns with pre-approved methods and details recorded within CMS and module handbooks. Schools will establish internal collection and quality review processes before distribution to the External Examiner for external scrutiny (see 5.1.1 and 5.2).

• Annual review by Programme/Subject Committees (or sub-groups) through annual monitoring meetings. Actions arising from meetings will be documented.

Programme/subject committees are responsible for ongoing review of the performance of programmes in the light of evidence available to them at programme and module level. Programme performance statistics to inform review by programme/subject teams are provided centrally by the Centre for Quality Enhancement.

Programme/subject committee annual monitoring meetings should consider all available information including student demographic data, statistical performance data, external examiner reports, PSRB and employer engagement, student feedback, DLHE and NSS results. These factors will influence further enhancement planning (via CA3) and/or the identification of good practice for dissemination.

Excerpt From Programme Approval Management & Review Handbook 2021 (pg 18):

Assessment quality considerations for internal review panels should include:

• Clear alignment of assessment strategies with quality frameworks and University strategic priorities.

• Clear alignment of assessment methods with module and programme level outcomes.

• Specific regulatory and professional requirements and exemptions to norms.

• The portfolio of assessments across the programme to ensure the mix of methods and delivery conditions are appropriate for the discipline area, while providing students with equal opportunities to demonstrate their learning and achievements.

• Accessibility of assessment methods across the programme and the quality of student guidance and support.

• The consistent provision of coursework briefs, marking schemes and rubrics and formative feedback.

• Consistency and quality of assessment information conveyed to students including format and clarity of briefs, marking schemes and rubrics, and the consistent quality of feedback. Assessment information must be free from typographical and/or calculation errors. Submission instructions should comply with Ulster expectations or identify that an approved exemption is in place.

• The level of assessment scaffolding provided to offer formative support and guidance for students to build their confidence and capability.

• Assessment size, workloads and equivalence, scheduling, and adherence to assessment parameters.

• The continued fair distribution of assessment tasks across the academic period.

• Reasonable adjustments procedures and communications.

• Currency of assessment information held on the Curriculum Management System and module database (see 5.1.1).

• Availability of a schedule of assessment dates by level and module (see appendix 2).

• Assessment performance, student achievement and risks of academic misconduct.

• The identification of innovative and effective practice for dissemination.

And for examinations, peer reviews should check:

• Appropriateness of formulation and clarity of questions.

• Appropriate coverage of module content and alignment with learning outcomes.

• Appropriateness and level of difficulty of questions aligned with level of study.

• Appropriate use of discriminating questions.

• The regular rotation of questions to avoid excessive repetition from previous papers.

• The appropriate number of questions for the time allocated.

• The appropriate weighting of questions and a clear indication to students of the marks available for each question.

• Accuracy of marking schemes and consideration of alternate responses to questions.

Revisions to previously approved programme or module assessment strategies will be undertaken through the programme revision process using the CA3 form. Proposed changes to assessment via CA3 will also prompt a review in the context of the wider programme to ensure scheduling, workloads and alignment with outcomes are not affected negatively.

The HoS will be responsible for ensuring in the event of ill-health or similar that students are provided with the appropriate assessment.

Assessment documentation for programmes offered by partner institutions may be reviewed by the Faculty Partnership Manager and/or a Faculty subject expert.

Further guidance:

• Course and Subject Management
• Continuous Assurance of Quality Enhancement

All summative assessments leading to an Ulster award or must undergo an internal school level scrutiny process prior to the submission of assessments to the External Examiner.

Each school must confirm with the relevant AD (AQSE) the mechanisms in place for the forthcoming year in terms of assessment scrutiny and security.

Schools may determine their preferred timeline for scrutiny, but it must take place sufficiently early in the academic calendar to ensure that modifications can identified and actioned in time. Approved assessment briefs must be available 48 hours before the first teaching session.

Assessment documentation for Externals Examiners must be pre-checked for the following:

Assessment documentation for programmes offered by partner institutions may be reviewed by the Faculty Partnership Manager and/or a Faculty subject expert.

Module Coordinators are responsible for proof-reading their draft exam papers and coursework assessment schemes and for complying with internal scrutiny processs (5.1). Heads of School will ensure assessment documentation is presented consistently and accurately to External Examiners.

All draft examination papers and coursework assessment schemes for the modules in each External Examiner’s area of responsibility are approved by the External Examiner in advance. Assessments should be drafted early enough in the academic cycle to allow sufficient time for the External Examiner to scrutinise and approve them before the module begins.

• Coursework assessment briefs and criteria/rubrics should be made available to the External Examiner in advance of the commencement of the presentation of a module
• Examination papers should be made available for review by week 3 (per relevant semester).
• Final formatted versions will be submitted to the Examinations Office by week 8

Supplementary (resit/referred) papers should be prepared at the same time as the main paper and submitted to the EE at the same time.

Draft assessments with accompanying marking schemes and, where appropriate, indicative points for content of answers are sent to the External Examiner for approval. The exact nature and extent of involvement must be discussed and agreed with External Examiners in advance. The External Examiner should be advised that, if comments are not received within three weeks, the University will assume that the draft is approved.

The External Examiner should be provided with an assessment schedule and all forms of assessment for each module. Documentation will be signed off by the Programme Director.

Communication from the External Examiner to review and implement any required changes should be via the School/Dept office, in consultation with Programme Director, and not with individual Module Coordinators. Module Coordinators will reflect upon recommendations and implement changes.

Copies of External Examiners comments will be supplied to Module Leaders. Required changes will be incorporated, recommendations will be considered and if necessary, discussed with the Programme/Subject Director or Associate Head of School.

Approval of a draft examination paper should be received from the External Examiner before it is submitted to the Examinations Office.

The Head of School is responsible for ensuring that all assessments once approved by the External Examiner are held securely.

Process for providing External Examiners A-code access to Blackboard, via A3M

Further information can be found on  the External Examiners webpages.