PAT Spelling Skills validation and updates 2024

As part of ACER’s standard quality assurance process, we have reviewed PAT Spelling Skills to validate its performance in assessing students’ spelling skills.

This article covers the following topics related to the validation exercise:

• Background
• Review
• Changes

Background

ACER puts great effort into pre-release trialling for new assessments, but post-release reviews of live test data are essential to ensure quality and identify any improvements that will strengthen the assessments. All new or updated assessments are reviewed 12 months after release or when sufficient data are available. PAT Spelling Skills was released in March 2022 and has seen considerable use by schools across Australia in that time.

Review

A complete check of all available PAT Spelling Skills results data was undertaken by ACER’s psychometric experts to identify any irregularities, while subject matter experts reviewed the test items themselves to ensure they were sound. The available data included nearly 100 000 completed tests from Australian students.

Changes

The key outcomes from the review are as follows:

• Difficulty scale scores have been adjusted for 40 test items (questions) across 14 PAT Spelling Skills test forms. These changes are reflected in updates to the average difficulty of the affected test forms.
• The changes in item difficulty scores also affect the calculation of students’ scale score achievement. 
• No changes were required to the way any items are scored.
• No test items have been changed or removed from the tests.

As a result of the changes to these item and test difficulty scores, students’ achieved scale scores have been re-calculated in the online reports. In the vast majority of cases, students' reported scale scores have changed by no more than ±2 scale score points, if at all.

In very rare cases, students’ reported scale scores may have changed by more than 2 points.

Online reports generated on or after 4 March 2024 display the updated scale scores for all students and items.

The average test difficulty scores before and after the changes were applied are shown in the table below.