The PAT Early Literacy reports help you understand students' abilities, diagnose learning gaps, andsupport growth in phonics, listening, and reading comprehension. Phonics results are diagnostic without a scale, while listening and reading comprehension use scale scores and achievement bands for comparison and growth tracking.
The following topics are covered here:
Related articles:
- Generating reports with the ACER Data Explorer
- Using the Achievement report (Listening and Reading Comprehension)
- Using the Item Performance report
- Using the Student report
- Using the Progress report
Phonics and Related Skills
Unlike other PAT assessments, PAT Early Literacy Phonics and Related Skills results are not aligned to a described achievement scale. This is because, while these skills are pre-requisites for future growth in reading comprehension, they are ‘constrained’ – growth in these skills stops once they are mastered. For this reason, assessment results are presented at the item, strand and strand process levels, allowing for diagnostic analysis to support you in understanding students’ needs.
Phonics and Related Skills results are reported at the item level within the Item Performance report in the ACER Data Explorer.
Several Phonics and Related Skills items are scored together as pairs. Students must correctly respond to both paired items to receive one score point. This approach ensures measurement reliability for two-option items.
Achievement bands and norm or reference comparison groups do not apply for Phonics and Related Skills assessments.
Listening Comprehension
PAT Early Literacy Listening Comprehension results are aligned to a new described scale, constructed as part of the development of PAT Early Literacy. Student achievement on the PAT Listening scale is measured in scale scores, which are explained in more detail below. Achievement bands - scale score ranges - are described in terms of the skills and knowledge typically expected of students achieving at that band.
As a new assessment, PAT Early Literacy Listening Comprehension does not have Australian reference data upon launch. Reference group data will be compiled and published when there are sufficient results data available.
Reading Comprehension
PAT Early Literacy Reading Comprehension aligns with the existing PAT Reading scale used by both PAT Reading 5th Edition and PAT Reading Adaptive. This allows for the monitoring of learning progress from the early years of school onward using PAT Reading achievement band descriptions.
PAT Reading Australian norms represent achievement of students nationally against the PAT Reading scale. Year 1 PAT Reading norms can be used as a reference point to understand the comparative achievement of your students.
Scale score
A scale score is a numerical value given to a student whose achievement has been measured by completing an assessment. A student's scale score lies at a point somewhere on the achievement scale, and it indicates that student's level of achievement in that particular learning area — the higher the scale score, the more able the student.
Regardless of the test level or items administered to students, they will be placed on the same scale for the learning area. This makes it possible to directly compare students' achievement and to observe students' progress within a learning area by comparing their scale scores from multiple testing periods over time.
A score on a Reading scale, for example, has no meaning on the Maths scale. In fact, the units of the scale will have different meanings for each scale. This is because these units are calculated based on the range of student levels of achievement, which vary widely between learning areas.
Achievement bands
While a scale score indicates a student’s achievement level and can be used to quantitatively track a student’s
growth, it is only in understanding what the number represents that teachers can successfully inform their practice to support student learning. For this reason, scales are divided into achievement bands that include written descriptions of what students are typically able to do at that band (band description).
Students in the same achievement band are operating at approximately the same achievement level within a learning area regardless of their school year level.
Viewing student achievement in terms of achievement bands may assist you to group students of similar abilities. By referencing the achievement band descriptions, you can understand the types of skills typical of students according to their band.
Item difficulty
Item difficulty is a measure of the extent of skills and knowledge required to be successful on the item. This makes it possible to allocate each test item a score on the same scale used to measure student achievement. An item with a high scale score is more difficult for students to answer correctly than a question with a low scale score. It could generally be expected that a student is able to successfully respond to more items located below their scale score than above.
Item difficulties are estimated based on the performance of individuals with a range of abilities who respond to that item, first at the item trial stage and later verified in real test results. The concept being assessed in the item is one aspect of item difficulty. Other factors may combine to make an item more or less complex. For example, the level of abstraction, the number of steps required, whether the question involves problem-solving or computation, the question context, the required precision of response, cognitive load, etc. An item assessing a concept that is introduced earlier in the curriculum may still be quite complex. Conversely, an item assessing a concept introduced later may be simpler.
By referencing the difficulty of an item, or a group of items, and the proportion of correct responses by a student or within a group, it may be possible to identify particular items, or types of items, that have challenged students.
Australian norms
Norm data that represents the achievement of students across Australia is available as a reference sample against which student achievement can be compared.
The comparison between a student's scale score achievement and the Australian norm sample can be expressed as a percentile rank.
The percentile rank of a score is the percentage of students who achieve less than that score. For example, a student with a percentile rank of 75th compared to the Year 3 norm sample has a scale score that is higher than 75% of Australian Year 3 students.