PAT STEM Contexts

The following topics are covered in this article:

• Assessment overview 
• When to assess 
• Choosing the right test 
• Administering the tests 
• Using the results 
• Supporting documents 

Assessment overview 

PAT Inquiry and Problem Solving in STEM Contexts (also referred to as PAT STEM Contexts) is an online assessment that measures students’ abilities to answer questions through inquiry and to solve problems in STEM contexts.

The overall purpose of STEM education is to enable individuals to use inquiry processes to draw conclusions and to solve problems in real-world contexts and situations. In today’s world, many questions or problems cannot be approached using knowledge or skills from a single discipline. Instead, individuals often need to draw together and integrate knowledge, skills and practices from multiple disciplines. When answering questions through inquiry within STEM, the focus is on building understanding of real-world phenomena and scientific investigations. When problem-solving, the focus is on finding solutions, which includes suggesting and evaluating actions, and proposing designs. Both inquiry questions and problems posed in STEM contexts require individuals to draw on their knowledge, skills and practices from across the four discipline areas of Science, Technology, Engineering and Mathematics.

In addition to providing a measure of students’ ability to answer questions using inquiry processes and to solve problems in STEM contexts, PAT Inquiry and Problem Solving in STEM Contexts may also support schools in conceptualising how to teach the discipline areas of Science, Technology, Engineering and Mathematics in an integrated way.

Some assessment items retain an emphasis on Science content descriptions across all three strands, allowing for learning to be assessed across Science Understanding; Science as a Human Endeavour; and Science Inquiry Skills strands. The majority of these items are also assessed in a real-world context, rather than addressing facts, knowledge and skills in isolation. Each item is also linked to one of the cognitive skills: Knowing, Applying and Reasoning, based on the classifications used in the Trends in International Mathematics and Science Study (TIMSS) 2019 assessment frameworks for Science and Mathematics.

There are two assessments available, with an emphasis on the middle years of schooling. The two assessments are Middle Years A (recommended for students in Years 4, 5 and 6) and Middle Years B (recommended for students in Years 6, 7 and 8).

The PAT Inquiry and Problem Solving in STEM Contexts assessments are delivered online to allow for the inclusion of dynamic stimulus and interactive response formats.

Stimulus materials for real-world contexts include animations, allowing for phenomena and processes to be presented in a dynamic manner. Students respond in a variety of interactive ways, including by use of drag-and-drop and hotspot functionality, as well as responding to simple multiple choice and complex multiple choice formats. There are also some items that require students to input a numerical response. Some items feature partial scoring; these are reported either as fully correct (2 score points), or partially correct (1 score point), providing further diagnostic information.

Why use PAT Inquiry and Problem Solving in STEM Contexts?

PAT Inquiry and Problem Solving in STEM Contexts has been developed to provide teachers with an easy-to-administer assessment that may help to fulfil one or more of the following purposes:

• Assessing aspects of students’ general capabilities in a STEM context
• Assessing the inquiry and problem solving in a practical and real-world context
• Diagnosing students’ strengths and weaknesses in relation to the key cognitive processes of knowing, applying and reasoning as applied in a STEM context
• Demonstrating the STEM inquiry and problem solving scenarios to students
• Raising the importance and profile of STEM
• Providing practical STEM tools and resources
• Measuring student progress in STEM related skills.

When to assess 

When to assess depends largely on the purpose for using the assessment. ACER envisages that most schools will want to measure students’ capabilities, use the data to inform teaching and then measure progress at a later date. For this purpose we suggest that the Middle Years A assessment might be administered in primary schools during Year 5, and Middle Years B administered during Year 6. In a secondary school we suggest administering the Middle Years A assessment at the beginning of Year 7 and the Middle Years B assessment during Year 8.

To monitor student progress, a gap of 9 to 12 months between testing sessions is recommended. Learning progress may not be reflected in a student’s PAT STEM Contexts scale scores over a shorter period of time.

Choosing the right test 

PAT STEM Contexts Middle Years A and Middle Years B assessments each have an emphasis on the Australian Curriculum outcomes at Years 5 and 6. Approximately half of the questions align to Years 5 and 6 outcomes. Middle Years A also contains questions aligned to Year 4; and Middle Years B also contains questions aligned to Years 7 and 8.

Based on the distribution of questions aligned to the curriculum, and on the average difficulty of the assessments, the suitable year levels for administration are Years 4, 5 and 6 for Middle Years A, and Years 6, 7 and 8 for Middle Years B.

To make decisions about which test is most appropriate for a particular student or group of students, it is essential that the teacher previews and becomes familiar with the content of the tests.

Administering the tests 

Prior to administering the tests to students, teachers should download or print a list of student login details as well as the test administration instructions. It is recommended that the school’s unique account URL is saved as a shortcut or link on students’ devices or on the school intranet for easy access.

Students’ focus and energy levels are important factors in their capacity to accurately demonstrate their ability on the assessment. For this reason, it is generally best to assess students in the morning and not immediately before or after an exciting school event.

 Test administration instructions

Each assessment begins with 9 practice items, followed by 34 items in the assessment.

Students have one hour to complete the assessment.

• Practice items: 5–10 minutes
• Assessment items: 45–50 minutes

Students can use the onscreen calculator provided as they complete the test.

Teachers of students with additional needs should assist according to the normal practice in their classroom. Any changes to standard testing conditions should be recorded for future reference.

The practice items

The practice items have been designed to model the navigation and item formats that appear in each assessment. The practice items are intentionally easy, so students are not distracted by the content. They allow students to practise responding to simple and complex multiple choice items, dragging and dropping objects, selecting a hotspot, using the onscreen calculator, watching an animation, and scrolling down the page to see some content.

Students can have unlimited teacher support for the practice items as response data for these questions is not part of the final score.

Once the practice items have been completed, students move on to complete the assessment independently.

Using the results 

Students’ results from both Middle Years A and Middle Years B tests are reported against a common PAT STEM Contexts scale, which ensures that teachers can identify attainment and progress. The scale scores relate specifically to the PAT Inquiry and Problem Solving in STEM Contexts assessment and are independent of other PAT scales including PAT Maths and PAT Science.

The PAT STEM Contexts scale has been categorised into five levels of achievement. Each level qualitatively describes the skills and understandings a student has demonstrated by responding to a PAT STEM Contexts assessment. The levels of achievement can also be used to compare student results obtained from different assessments and assessed at different times. The PAT STEM Contexts scale and achievement band descriptions were developed following rigorous item trialling and analysis of assessment items and the assessment construct.

Group Report

The Group Report displays question-level information about student achievement.

PAT Inquiry and Problem Solving in STEM Contexts is the first online PAT assessment to utilise partial-score questions, for which students may achieve 0, 1 or 2 score points according to the accuracy of their responses. To accommodate this, the Group Report for this assessment differs slightly from that of other online PAT assessments.

A summary of the key features of the PAT Inquiry and Problem Solving in STEM Contexts Group Report is included below.

Click the images to view full-size.

For each question in PAT Inquiry and Problem Solving in STEM Contexts, the Group Report displays the following information: