PAT STEM Contexts

The following topics are covered in this article:

• Assessment overview 
• When to assess 
• Choosing the right test 
• Administering the tests 
• Using the results 
• Supporting documents 

Assessment overview 

PAT Inquiry and Problem Solving in STEM Contexts (also referred to as PAT STEM Contexts) is an online assessment that measures students’ abilities to answer questions through inquiry and to solve problems in STEM contexts.

The overall purpose of STEM education is to enable individuals to use inquiry processes to draw conclusions and to solve problems in real-world contexts and situations. In today’s world, many questions or problems cannot be approached using knowledge or skills from a single discipline. Instead, individuals often need to draw together and integrate knowledge, skills and practices from multiple disciplines. When answering questions through inquiry within STEM, the focus is on building understanding of real-world phenomena and scientific investigations. When problem-solving, the focus is on finding solutions, which includes suggesting and evaluating actions, and proposing designs. Both inquiry questions and problems posed in STEM contexts require individuals to draw on their knowledge, skills and practices from across the four discipline areas of Science, Technology, Engineering and Mathematics.

In addition to providing a measure of students’ ability to answer questions using inquiry processes and to solve problems in STEM contexts, PAT Inquiry and Problem Solving in STEM Contexts may also support schools in conceptualising how to teach the discipline areas of Science, Technology, Engineering and Mathematics in an integrated way.

Some assessment items retain an emphasis on Science content descriptions across all three strands, allowing for learning to be assessed across Science Understanding; Science as a Human Endeavour; and Science Inquiry Skills strands. The majority of these items are also assessed in a real-world context, rather than addressing facts, knowledge and skills in isolation. Each item is also linked to one of the cognitive skills: Knowing, Applying and Reasoning, based on the classifications used in the Trends in International Mathematics and Science Study (TIMSS) 2019 assessment frameworks for Science and Mathematics.

There are two assessments available, with an emphasis on the middle years of schooling. The two assessments are Middle Years A (recommended for students in Years 4, 5 and 6) and Middle Years B (recommended for students in Years 6, 7 and 8).

The PAT Inquiry and Problem Solving in STEM Contexts assessments are delivered online to allow for the inclusion of dynamic stimulus and interactive response formats.

Stimulus materials for real-world contexts include animations, allowing for phenomena and processes to be presented in a dynamic manner. Students respond in a variety of interactive ways, including by use of drag-and-drop and hotspot functionality, as well as responding to simple multiple choice and complex multiple choice formats. There are also some items that require students to input a numerical response. Some items feature partial scoring; these are reported either as fully correct (2 score points), or partially correct (1 score point), providing further diagnostic information.

Why use PAT Inquiry and Problem Solving in STEM Contexts?

PAT Inquiry and Problem Solving in STEM Contexts has been developed to provide teachers with an easy-to-administer assessment that may help to fulfil one or more of the following purposes:

• Assessing aspects of students’ general capabilities in a STEM context
• Assessing the inquiry and problem solving in a practical and real-world context
• Diagnosing students’ strengths and weaknesses in relation to the key cognitive processes of knowing, applying and reasoning as applied in a STEM context
• Demonstrating the STEM inquiry and problem solving scenarios to students
• Raising the importance and profile of STEM
• Providing practical STEM tools and resources
• Measuring student progress in STEM related skills.

When to assess 

When to assess depends largely on the purpose for using the assessment. ACER envisages that most schools will want to measure students’ capabilities, use the data to inform teaching and then measure progress at a later date. For this purpose we suggest that the Middle Years A assessment might be administered in primary schools during Year 5, and Middle Years B administered during Year 6. In a secondary school we suggest administering the Middle Years A assessment at the beginning of Year 7 and the Middle Years B assessment during Year 8.

To monitor student progress, a gap of 9 to 12 months between testing sessions is recommended. Learning progress may not be reflected in a student’s PAT STEM Contexts scale scores over a shorter period of time.

Choosing the right test 

PAT STEM Contexts Middle Years A and Middle Years B assessments each have an emphasis on the Australian Curriculum outcomes at Years 5 and 6. Approximately half of the questions align to Years 5 and 6 outcomes. Middle Years A also contains questions aligned to Year 4; and Middle Years B also contains questions aligned to Years 7 and 8.

Based on the distribution of questions aligned to the curriculum, and on the average difficulty of the assessments, the suitable year levels for administration are Years 4, 5 and 6 for Middle Years A, and Years 6, 7 and 8 for Middle Years B.

To make decisions about which test is most appropriate for a particular student or group of students, it is essential that the teacher previews and becomes familiar with the content of the tests.

Administering the tests 

The Test administration instructions linked at the bottom of this page provide further detail, including teacher scripts and troubleshooting tips.

Each assessment begins with 9 practice items, followed by 34 items in the assessment.

Students have one hour to complete the assessment.

• Practice items: 5–10 minutes
• Assessment items: 45–50 minutes

Students can use the onscreen calculator provided as they complete the test.

Teachers of students with additional needs should assist according to the normal practice in their classroom. Any changes to standard testing conditions should be recorded for future reference.

Preparation

The following steps need to be completed ahead of time:

• Check the technical requirements and run the browser exam from a student device to identify any potential technical issues.
• Schedule your testing date and time. It is best to administer tests in the morning and not immediately before or after an exciting school event.
• Ensure that all students are listed within your school's online assessment account and have been assigned the necessary tests.
• Review and assign appropriate test levels to all students according to their ability (non-Adaptive tests only).
• Download or print a list of your students' login details.
• Make note of your school's online assessment login page, or make sure that the URL is saved on student devices, or available to your students as a link. The address will be similar to https://oars.acer.edu.au/your-school-name.

Instructions

• Tests should be administered under standard testing conditions with invigilation.
• Students' screens should be monitored as part of test invigilation.
• Students are permitted to use pen/pencil and paper to make notes during the test.

If you determine that some students require changed testing conditions due to specific learning needs, these changes should be recorded for future reference. The process for determining and implementing any changes to test conditions should be consistent between classes and across the school.

Delivery

• Students will not be automatically locked out of the tests after the allowed time passes. You must monitor and manage the time, including accommodating toilet breaks or other interruptions that may occur.
• Student responses are automatically saved each time they navigate to another question.
• If technical problems cause the need to postpone the completion of the tests, students may close the browser without losing their progress.
• iPads and tablet devices must be held in landscape orientation.

Although PAT is low-pressure and low-stakes, the following new security features have been designed to support you by restricting undesirable actions during test-taking.

• No installation required – the features run directly in the browser across Windows, Mac, iPad and Chromebook.
• Full-screen browser window – test starts with the browser in full-screen mode to encourage full student focus on the test content
• Warnings – if the student attempts to exit full-screen mode or leave the browser window during the test, the system responds by blurring the test screen and displaying a warning to the student.

The practice items

The practice items have been designed to model the navigation and item formats that appear in each assessment. The practice items are intentionally easy, so students are not distracted by the content. They allow students to practise responding to simple and complex multiple choice items, dragging and dropping objects, selecting a hotspot, using the onscreen calculator, watching an animation, and scrolling down the page to see some content.

Students can have unlimited teacher support for the practice items as response data for these questions is not part of the final score.

Once the practice items have been completed, students move on to complete the assessment independently.

Using the results 

Students’ results from both Middle Years A and Middle Years B tests are reported against a common PAT STEM Contexts scale, which ensures that teachers can identify attainment and progress. The scale scores relate specifically to the PAT Inquiry and Problem Solving in STEM Contexts assessment and are independent of other PAT scales including PAT Maths and PAT Science.

The PAT STEM Contexts scale has been categorised into five levels of achievement. Each level qualitatively describes the skills and understandings a student has demonstrated by responding to a PAT STEM Contexts assessment. The levels of achievement can also be used to compare student results obtained from different assessments and assessed at different times. The PAT STEM Contexts scale and achievement band descriptions were developed following rigorous item trialling and analysis of assessment items and the assessment construct.

Group Report

The Group Report displays question-level information about student achievement.

PAT Inquiry and Problem Solving in STEM Contexts is the first online PAT assessment to utilise partial-score questions, for which students may achieve 0, 1 or 2 score points according to the accuracy of their responses. To accommodate this, the Group Report for this assessment differs slightly from that of other online PAT assessments.

A summary of the key features of the PAT Inquiry and Problem Solving in STEM Contexts Group Report is included below.

Click the images to view full-size.

For each question in PAT Inquiry and Problem Solving in STEM Contexts, the Group Report displays the following information: