A robotic agent (‘Karel’) in the shape of a turtle needs help to navigate and collect/place stones in a grid-like ‘world’. Students will be asked to learn and apply computational problem-solving concepts and skills in a block-based programming environment to ensure Karel can achieve its goals. 

To get started, you can assign your students one of our expert-designed Assessment Experiences (maps), build a new Assessment Experience (map) using the PILA assignment tool, or design an individual task using the Karel task customiser tool. 

PILA_KarelApp_description_June 2021.pdf

Welcome to the Karel assessment application! 'Karel', a turtle-shaped robotic agent, needs help to navigate this grid-like ‘world’ and complete a few missions. To ensure Karel can achieve its goals, students will learn and apply computational problem-solving concepts and skills in a simple, accessible block-based programming environment.

Karel the Turtle

Recognizing that no class is the same and that a given library of exercises may not always satisfy specific learning goals, PILA provides you with intuitive, easy-to-use tools to create assessment experiences and tasks that suit your students' needs and skills to the best.

Using the assignment tool, create a new assessment experience (map) targeted to your students’ needs by selecting and organising tasks developed by PILA experts, you, or other teachers/researchers. 

Build your own assessment experiences (maps)

With no programming experience required, this task customiser tool allows you to design new tasks for your students to try out. You can either modify an existing task, or start fresh. 

PILA | Welcome to the Karel assessment application

PILA | Karel the turtle

Karel the Turtle is an application where students are asked to learn and apply concepts and skills specific to computational problem solving while helping Karel the turtle navigate his ‘world’ in a block-based coding environment. This application challenges students with assessment experiences or maps that are either 

1) pre-developed by PILA experts and piloted in schools, or 

2) created by YOU or other fellow teachers with the help of simple PILA customisation tools. 

The best part? No programming skills are needed!

Traditional text-based programming languages can be difficult to learn, especially for beginners. Block-based coding, like ones seen on our Karel the Turtle application, gives learners the opportunity to develop basic computational thinking skills that are essential for programming.

Click on the blue button to access & browse the complete Karel assessment experiences (maps) and tasks library. No map or task perfectly suits your teaching needs? You can create your own! Scroll down tp learn more about the the Karel Customizer tools.

The assessment experiences (maps) are developed by experts to provide a snapshot of where students are in their learning. All assessment experiences begin with a short tutorial, followed by a series of shorter tasks that are meant to prepare students for a longer challenge task, and end with a brief questionnaire. 

To assign an assessment experience (map), click on "Assign". A link will be created that you can share with your students. A new session will also appear in your account, corresponding to the assessment experience (map) you assigned: here you will find information on student performance. 

You can access the Karel assessment experiences (map) and tasks library by clicking on the blue button below.

Coming in 2022  - In this application, students will be assessed on their ability to learn and subsequently teach a virtual agent named Betty about a complex phenomenon. In order to do this well, students will need to study resources and then construct an accurate causal model of the systems or processes that make up the complex phenomenon. To receive feedback on the completeness and accuracy of their understanding, students can ask Betty to answer questions based on the information they conveyed through their causal map. 

While we work on Betty's Brain, you can...

Help Betty do well on her test! Learn about complex phenomena, uncover and graphically represent the interaction among different variables and teach Betty about it all. Coming in early 2022.

Betty's Brain

COMING IN 2023 - Created by Enlightware, Cand.li unites creative thinking and computational problem solving by letting users design and build videogames from scratch using simple visual programming. On PILA, Cand.li challenges students to beat game-like levels by placing missing platforms, building intricate mechanisms and coding new movements and interactions for their character. 

COMING IN 2023 - Cand.li unites creative thinking and computational problem solving by letting users design and build games from scratch using simple visual programming.

This website uses cookies to improve your browsing experience.

Have a look around while we work on Betty's Brain!

See more content from around the PILA website

Introduction

Application guide

PILA assessment application for computational problem solving

‘Karel’ is a PILA application that measures students’ computational problem solving skills. The application asks students to build programs in a block-based visual programming interface in order to instruct a robot in the form of a turtle (“Karel”) to perform certain actions. Karel understands four basic instructions: “move forward”, “turn left”, “place stone”, and “pick-up stone”, in a grid-like world. These programmed actions can be seen as abstractions of various navigational problems in the real world.  

In the Karel  application, students will first complete a tutorial and a series of short, scaffolded tasks where they are introduced to the environment (i.e. visual programming interface) and learn some of the basic concepts involved in computational problem solving. Students then progress to more complex and open-ended challenges where they have to apply simultaneously multiple skills and concepts.

The Karel application has a “low floor and high ceiling”, meaning students who have minimal programming knowledge and/or computational thinking skills can nevertheless productively interact with the environment because of its intuitiveness and simplicity (i.e. low floor), while at the same time this same environment allows for the design of high complexity tasks that can challenge even expert programmers (i.e. high ceiling). The application accomplishes this by using a block-based programming environment.

The Karel assessment application is composed of a visual block-based programming interface and the ‘task hub’. Figures 1 and 2 show the current design of these two interfaces.

Figure 1. Block-based programming interface                                                                                                                                                                                          

1. Block-based program builder: Students can drag and drop pre-defined blocks of code into the program builder in order to build programs that instruct Karel. Students will have access to a certain set of block types and functionalities according to the design of a given task.                                                                          

 2. World state: This is the grid-like world within which Karel the robot lives and executes actions. Each cell in the grid can be empty or contain a finite amount of stones. Cells can be separated by a wall that blocks Karel from moving forward. The world state shows the current arrangement of stones and walls, and the position and orientation of Karel. The program that students build and ‘run’ will result in the world state changing, modelling Karel’s behaviour in the world.        

3. Goal state: This is the desired state of the world, which is defined by the arrangement of stones and walls, and the position and orientation of Karel.           

4. ‘Run’ button: The run button allows students to execute the program in the program builder as a simulation of Karel’s behaviour in the ‘world’.                                      

5. Pop-up messages: These messages automatically appear after the student tests a program to communicate errors (e.g. ‘front is blocked’ or ‘your program is empty’) or provide feedback (e.g. ‘not quite’ or ‘success’) to the student.                                           

6. Task instructions: Task instructions succinctly state what students must do in the task and what the desired goal is. The goal can also be represented visually, displaying the desired final state of the world upon task completion (see #3).               

7.  Home button: This button allows students to return to the task hub (see Figure 2 for an example of the task hub). 

8. ‘Worked examples’ tabs: The worked examples illustrate a similar task, providing one ‘good’ example that demonstrates a possible approach to solving the problem and one ‘bad’ example that demonstrates a typical mistake. 

9. ‘Hint’ button: The hint button provides students with a hint system guiding students on how to successfully progress through the task. 

10. Time remaining clock: This provides students with an indication of how much time remains to complete the task. 

The ‘task hub’ (Figure 2) allows students to navigate between tasks. As students complete tasks, new tasks become unlocked. Learning tasks (e.g. tasks 1-6 from the top in Figure 2) aim to introduce specific concepts and help develop the target skills throughout this ‘phase’. Once the students complete the targeted number of learning tasks, the challenge task will be unlocked. The challenge task is a complex, open-ended task that will ask students to apply what they practices in the previous tasks. 

Key elements of the digital environment

Figure 2. Task Hub                                                                                                                                                                                                                                          

The tasks in the Karel application have been designed to provide evidence on the computational problem solving skills described in the PILA Computational Problem Solving Framework.  Each task is mapped to a given cell in the evaluation rubric presented in the Framework. Students’ capacity to solve a given task in Karel is considered evidence that they can do what is described in the corresponding cell of the rubric.  

Here are a few examples of the evidence that can be collected from Karel for each of the skills:

The student can identify and put in the right order relevant sub-goals for a given problem from given lists of sub-goals 

The student can use a drawing tool to indicate different trajectories that Karel can follow in the world to collect all the stones*

The   student solves a problem that requires using nested repetitions

The   student can tag repeating patterns in the world using a drawing tool*. 

The student can design a program that achieves the goal state in   multiple different worlds

The student saves functions in a function bank and uses them across   different tasks*

 The student can fix an erroneous program

 The student tests the program in chunks as they work 

What skills can we assess with Karel?

What information can teachers expect on how students manage their learning and stay engaged?

In addition to demonstrating their proficiency with computational problem solving skills and associated concepts (sequences, conditionals, loops and functions), students will need to be capable of effectively regulating their work on the digital platform, and maintaining an adequate level of engagement during the assessment experience. 

We provide an indication for the information that will be collected on the way in which students:

The student accesses a worked example after   making at least one attempt to build a program

The student requests a hint after attempting   to solve the problem

 The student performs actions that are consistent with the automated   feedback (for example, the student reduces the number of blocks in the   program after receiving the feedback that the program could be more   efficient) 

The student correctly evaluates his/her own performance in the   challenge task.

The student does not remain inactive for long periods of time (e.g.   more than one minute at a time)

The student reports feeling engaged during the task

What information can teachers expect?

There are three principal ways in which teachers can use the Karel application, which include:

Assessment experiences defined by experts that teachers can assign to their class to complete during classtime or as  homework. These experiences typically last around one hour. They contain a meaningful sequence of tasks that make students practice a defined set of skills and concepts, and progressively increase in difficulty. Teachers      can assign their students multiple assessment experiences, possibly      intervalled with other teaching practices (e.g. additional lessons, class discussion, etc.), and examine the extent to which students make progress from one experience to the next.   


Assessment experiences defined by teachers with the PILA Assignment tool. This tool allows teachers to combine discrete tasks that have been developed by experts,  other teachers/researchers, or the teacher themselves (using the Karel Task Customiser tool; see #3 below), in order to create tailored assessment experiences for their students. Teachers can design new assessment experiences that, for example, have      alternative sequences of tasks of varied difficulty, focus on one specific concept or skills, or span a desired length of time (e.g. a 20-minute experience that comprises four 5-minute discrete tasks). 


Assessment tasks defined by teachers. The Karel Task Customiser tool allows teachers with no programming experience to create their own task by modifying an existing task. By using the tool, teachers can modify several parameters, such as the task instructions, the world and goal states, and available learning resources. Any new task that is created through the customiser tool will automatically contain the necessary data collection capabilities, ensuring information on student      performance can be displayed in the digital dashboards for teachers and students to review. 

The tasks in Karel are classified according to the computational problem solving skills they target, their level of complexity, as well as the task type (see below), creator (e.g. PILA expert) and the characteristics of a solution that an expert would formulate. Each task created in the module has to include an ‘expert’ solution, and this section of the metadata is populated automatically extracting features of the expert solution. Task authors can also provide a short description of the task.

How can teachers use Karel?

The PILA computational problem solving module includes several types of tasks designed to assess computational problem solving as defined in the framework. Here is a short description of each task type in this module, and examples that are being implemented in Karel World:

Open-response or multiple choice questions

With a given program and initial world state, the student is asked to predict the goal state;

With a given initial world state and goal state, the student is asked to infer from a list of programs which program would produce the goal state;

With a given initial world state and “functioning” program for one goal state, the student is prompted to select from a set of alternative goal states the one(s) that can be accomplished with the same program;

With a given erroneous program, the student is asked to identify one (or more) errors; 

With a given program with numbered lines of blocks, the student is asked to describe what a selected set of lines accomplishes in the world.

Build a program from scratch with a combination of the following affordances:
      · Move Karel;
      · Pick up/Place stone;
      · Repetition block (run the inside function [x] times);
      · While statement;
      · If/else statement;
      · Create a custom function (with a label)

Modify a program to adjust to different goal states in the same or different worlds.

Parson’s problem  - All of the blocks of the program and the number of times they need to be used are defined but the blocks are not arranged in any specific order. Students are asked to arrange the blocks to create a functioning program. The task may include visual blocks or natural language to represent the discrete steps. 

Draw in the world  - Students are prompted to decompose the problem visually, for example by drawing walls around certain regions of the grid, or by drawing the shortest path that Karel can follow to collect all stones.

Black box simulations  - With a given “black box” function (i.e. a function block that students can use but that does not reveal what it does), students are asked to use the Karel environment and additionally available blocks to determine what the function does. 

Adapt Karel worlds  - With an erroneous program, world state and goal state, the student is asked to modify the goal state (e.g. adding/removing stones, adding/removing walls, changing orientation of Karel, etc.) so the given program is successful.

Karel task types

Modifiable task parameters in the customiser tool

For each of the task types, there are several parameters with which tasks are defined. The Karel Task Customiser tool provides teachers the opportunity to create or adapt tasks by modifying the following parameters, which are enumerated in Figure 4 below: 

Figure 4. Example of the authoring interface for the Karel Task Customiser tool

1. Position of Karel: The teacher can modify the position and direction that Karel is facing in the world and goal states.

2. Area of world and goal states: The world and goal states can be adjusted to contain a specific number of rows and columns.

3. Stones: The number and position of stones can be modified for both the initial world state and goal state. 

4. Walls: Walls that constrain the movement of the agents in the world can be added, removed and/or reoriented. 

5. Available blocks: Teachers can designate which blocks students can access and use, and how many of each block they can use.

6. Provision of initial program fragments: A partial segment of a program can be provided to students to complete. Teachers can adjust the segment      according to the aim and target difficulty of the task. 

7. Time limit: Teachers can decide to limit the amount of time students have to solve the problem.      

8. Number of tests: Teachers can decide to restrict the number of tests available for students to use when building their solution.

9. Number of command blocks: Teachers can limit the number of blocks students can use in a program. 

10. Inclusion of ‘contrasting cases’: Tasks can include examples of worked solutions to similar problems. Two types of examples can be provided: a      ‘good’ example, showing a correct and efficient way to solve a similar problem, and a ‘bad’ example, showing a typical mistake that the student      should avoid. 

11. Trigger-based hints: Students can click on a help button to receive a hint. The hint system can have up to two-levels with recommended content: the first level provides a general prompt, while the second level provides more specific information about how to solve the given problem. Teachers will be able to adjust the content of the hints or decide to also deactivate the hint feature.


Reporting dashboards in the Karel application

Students’ performance in Karel will be communicated to teachers and students through a digital dashboard. The dashboard includes multiple sections that summarise student performance at various levels of granularity, as well as allow teachers to replay students’ work and review their progress in a given task.  Indicators are populated in real-time using information from learner responses and interaction data. The dashboard aims to help teachers respond to several questions, such as: 

·        What problems are the students capable of solving?  

·        What computational problem solving skills do the students master or struggle with?

·        How capable are students to develop well-organised and readable programs?

·        How are students making progress when they are stuck, and how do they manage their work? 

·        What new task and/or new task sequences can provide more information on my students’ skills? 

The dashboard is organised in four different tabs providing information at different levels of granularity.  

The class summary (Figure 5) is the default view. It provides an easy-to-read overview of the performance of all students across the different tasks within a particular session. This summary shows task completion (a green cell for completed task, a red cell for incomplete task) for each task in the session. For the more complex tasks, it is also possible to display an indication of how far the students went in solving the task (see the column for task 5).  

Reporting dashboards

Figure 5. Example of the performance dashboard: Class view 

The student summary describes the performance of each individual student, indicating how the student performed with respect to the evaluation rubric in the PILA computational problem solving framework. The summary shows what basic, intermediate and expert problems the students was able to solve, for each of the four computation problem solving skills. In the example shown in figure 6, the student Xiaver was able to solve basic and intermediate problems in which he had to demonstrate decomposition, pattern recognition and debugging skills. However, Xiaver was not able to solve an expert task that involved pattern recognition and generalisation. The limited evidence from a short sequence of five tasks indicates that Xiaver has thus not yet reached an expert level in any of the skills. The summary also describes students’ programming behaviour, as evidenced by his use of control flows and testing. In the example, Xiaver had to solve three tasks that required using loops or conditional to address repeating patterns, and the dashboard shows that he used these control flow structures in the three tasks.         

Figure 6. Example of the performance dashboard: Student summary view

In the task summary, teachers can get more in depth information on what each student did on each task. Teachers can browse several indicators, that are grouped in four categories. The indicators in the ‘Activity’ category show how students used their time on the task, including their overall time on the task, the time to first action, the time spent inactive and the number of attempts they made. The ‘Control flows’ category indicates whether students included loops, conditionals or functions in their solution. The ‘Support’ category includes indicators on students’ use of the different scaffolds provided in Karel (hints and worked examples), and whether they responded to the automated feedback. Finally, the ‘Self-evaluation’ indicators summarise responses to survey questions at the end of the assessment that ask the students to evaluate their achievement and report on their engagement and feelings during the experience.  

Figure 7. Example of the performance dashboard: Task summary view

In this section of the dashboard, the teacher can select a task and a student to replay his/her work on the task, as well as visualise the progress the student made to solve the task (see Figure 8). The teacher can replay particular phases of the students’ work by dragging the timer along the time elapsed bar. They are guided in the selection of what to replay by a progress chart, that shows how much and how fast students progressed towards a functional solution, and the timing of other actions, such as conducting tests or browsing examples.  

Figure 8. Example of the performance dashboard: Progress and replay view

Who is behind PILA’s Karel the Turtle?

‘Karel World’ is a PILA application that measures students’ computational problem solving skills. The application asks students to build programs in a block-based visual programming interface in order to instruct a robot in the form of a turtle (“Karel”) to perform certain actions. This page will guide you through the Karel Application features and how you can best integrate it into the classroom.

Karel Application Guide

‘Computational problem solving’  is the iterative process of developing  computational solutions to problems. Computational solutions are expressed as logical sequences of steps (i.e. algorithms), where each step is precisely defined so that it can be expressed in a form that can be executed by a computer. Much of the process of computational problem solving is thus oriented towards finding ways to use the power of computers to design new solutions or execute existing solutions more efficiently. 

Using computation to solve problems requires the ability to think in a certain way, which is often referred to as ‘computational thinking’. The term originally referred to the capacity to formulate problems as a defined set of inputs (or rules) producing a defined set of outputs. Today, computational thinking has been expanded to include thinking with many levels of abstractions (e.g. reducing complexity by removing unnecessary information), simplifying problems by decomposing them into parts and identifying repeated patterns, and examining how well a solution scales across problems. 

Why is computational problem solving important and useful?

Computers and the technologies they enable play an increasingly central role in jobs and everyday life. Being able to use computers to solve problems is thus an important competence for students to develop in order to thrive in today’s digital world. Even people who do not plan a career in computing can benefit from developing computational problem solving skills because these skills enhance how people understand and solve a wide range of problems beyond computer science. 

This skillset can be connected to multiple domains of education, and particularly to subjects like science, technology, engineering or mathematics (STEM) and the social sciences. Computing has revolutionised the practices of science, and the ability to use computational tools to carry out scientific inquiry is quickly becoming a required skillset in the modern scientific landscape. As a consequence, teachers who are tasked with preparing students for careers in these fields must understand how this competence develops and can be nurtured. At school, developing computational problem solving skills should be an interdisciplinary activity that involves creating media and other digital artefacts to design, execute, and communicate solutions, as well as to learn about the social and natural world through the exploration, development and use of computational models. 

Is computational problem solving the same as knowing a programming language? 

A programming language is an artificial language used to write instructions (i.e. code) that can be executed by a computer. However, writing computer code requires many skills beyond knowing the syntax of a specific programming language. Effective programmers must be able to apply the general practices and concepts involved in computational thinking and problem solving. For example, programmers have to understand the problem at hand, explore how it can be simplified, and identify how it relates to other problems they have already solved. Thus, computational problem solving is a skillset that can be employed in different human endeavours, including programming. When employed in the context of programming, computational problem solving ensures that programmers can use their knowledge of a programming language to solve problems effectively and efficiently.  

Students can develop computational problem solving skills without the use of a technical programming language (e.g. JavaScript, Python). In the PILA module, the focus is not on whether students can read or use a certain programming language, but rather on how well students can use computational problem solving skills and practices to solve problems (i.e. to “think” like a computer scientist). 

How is computational problem solving assessed in PILA? 

Computational problem solving is assessed in PILA by asking students to work through dynamic problems in open-ended digital environments where they have to interpret, design, or debug computer programs (i.e. sequences of code in a visual format). PILA provides ‘learning assessments’, which are assessment experiences that include resources and structured support (i.e. scaffolds) for learning. During these experiences, students iteratively develop programs using various forms of support, such as tutorials, automated feedback, hints and worked examples. The assessments are cumulative, asking students to use what they practiced in earlier tasks when completing successive, more complex tasks. 

To ensure that the PILA module focuses on foundational computational problem solving skills and that the material is accessible to all secondary school students no matter their knowledge of programming languages, the module includes an assessment application, ‘Karel World’, that employs an accessible block-based visual programming language. Block-based environments prevent syntax errors while still retaining the concepts and practices that are foundational to programming. These environments work well to introduce novices to programming and help develop their computational problem solving skills, and can be used to generate a wide spectrum of problems from very easy to very hard. 

What is assessed in the PILA module on computational problem solving?

The module assesses the following set of complementary problem solving skills, which are distinct yet are often used together in order to create effective and efficient solutions to complex problems:

Decomposition is the act of breaking down a problem goal into a set of smaller, more manageable sub-goals that can be addressed individually. The sub-goals can be further broken down into more fine-grained sub-goals to reach the granularity necessary for solving the entire problem.

Pattern recognition refers to the ability to identify elements that repeat within a problem and can thus be solved through the same operations. Adressing repeating patterns means instructing a computer to iterate given operations until the desired result is achieved. This requires identifying the repeating instructions and defining the conditions governing the duration of the repetition.

Generalisation is the thinking process that results in identifying similarities or common differences across problems to define problem categories. Generalising solution results in producing programs that work across similar problems through the use of ‘abstractions’, such as blocks of organised, reusable sequence(s) of instructions. 

Solving a complex computational problem is an adaptive process that follows iterative cycles of ideation, testing, debugging, and further development. Computational problem solving involves systematically evaluating the state of one’s own work, identifying when and how a given operation requires fixing, and implementing the needed corrections. 

In order to apply these skills to the programming tasks presented in the module, students have to master the below set of programming concepts. These concepts can be isolated but are more often used in concert to solve computational problems:

Sequences are lists of step-by-step instructions that are carried out consecutively and specify the behavior or action that should be produced. In Karel World, for example, students learn to build a sequence of block commands to instruct a turtle to move around the world, avoiding barriers (e.g. walls) and performing certain actions (e.g. pick up or place stones). 

Conditional statements allow a specific set of commands to be carried out only if certain criteria are met. For example, in Karel World, the turtle can be instructed to pick up stones ‘if stones are present’.

To create more concise and efficient instructions, loops can communicate an action or set of actions that are repeated under a certain condition. The repeat command indicates that a given action (i.e. place stone) should be repeated through a real value (i.e. 9 times). A loop could also include a set of commands that repeat as long as a Boolean condition is true, such as ‘while stones are present’. 

Creating a function helps organise a program by abstracting longer, more complex pieces of code into one single step. By removing repetitive areas of code and assigning higher-level steps, functions make it easier to understand and reason about the various steps of the program, as well as facilitate its use by others. A simple example in Karel World is the function that instructs the turtle to ‘turn around’, which consists of turning left twice. 

Programming concepts

How is student performance evaluated in the PILA module?

Student performance in the module is evaluated through rubrics. The rubrics are structured in levels, that succinctly describe how students progress in their mastery of the computational problem solving skills and associated concepts. The levels in the rubric (see Table 1) are defined by the complexity of the problems that are presented to the students (simple, relatively complex or complex) and by the behaviours students are expected to exhibit while solving the problem (e.g., using functions, conducting tests). Each problem in the module is mapped to one or more skills (the rows in the rubric) and classified according to its complexity (the columns in the rubric). Solving a problem in the module and performing a set of expected programming operations thus provide evidence that supports the claims about the student presented in the rubric. The more problems at a given cell of the rubric the student solves, the more conclusive is the evidence that the student has reached the level corresponding to that cell.  

Please note: the rubric is updated as feedback is received from teachers on the clarity and usefulness of the descriptions.

Table 1. Rubric for computational problem solving skills

The performance of students on the PILA module depends not just on their mastery of computational problem solving skills and concepts, but also on their capacity to effectively manage their work in the digital learning environment. The complex tasks included in the module invite students to monitor, adapt and reflect on their understanding and progress. The assessment will capture data on students’ ability to regulate these aspects of their own work and will communicate to teachers the extent to which their students can: 

PILA tasks provide resources such as worked examples that students can refer to as they build their own solution. Students use resources effectively when they recognise that they have a knowledge gap or need help after repeated failures and proceed to accessing a learning resource. 

As students work through a PILA assessment, they receive different types of automated feedback (e.g.: ‘not there yet’, ‘error: front is blocked’, ‘try using fewer blocks’). Students who can successfully adapt are able to perform actions that are consistent with the feedback, for example inserting a repetition block in their program after the feedback ‘try using fewer blocks’.

In the assessment experiences designed by experts in PILA, the final task is a complex, open challenge. Upon completion of this task, students are asked to evaluate their own performance and this self-assessment is compared with their actual performance on the task.

The assessment will also collect information on the extent to which students are engaged throughout the assessment experience. Evidence on engagement is collected through questions that are included in a survey at the end of the assessment, and through information on students’ use of time and number of attempts.  

What additional information can be gleaned from the assessment experiences?

Learn about computational problem solving-related learning trajectories: 

Rich, K. M., Strickland, C., Binkowski, T. A., Moran, C., & Franklin, D. (2017). K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals. Proceedings of the 2017 ACM Conference on International Computing Education Research, 182–190.   

Rich, K. M., Strickland, C., Binkowski, T. A., & Franklin, D. (2019). A K-8 Debugging Learning Trajectory Derived from Research Literature. Proceedings of the 50th ACM Technical Symposium on Computer Science Education, 745–751. https://doi.org/10.1145/3287324.3287396

Rich, K. M., Binkowski, T. A., Strickland, C., & Franklin, D. (2018). Decomposition: A K-8 Computational Thinking Learning Trajectory. Proceedings of the 2018 ACM Conference on International Computing Education Research  - ICER ’18, 124–132. https://doi.org/10.1145/3230977.3230979 

Learn about the connection between computational thinking and STEM education: 

Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2015). Defining Computational Thinking for Mathematics and Science Classrooms. Journal of Science Education and Technology, 25(1), 127–147. doi:10.1007/s10956-015-9581-5

Learn how students apply computational problem solving to Scratch: 

Brennan, K., & Resnick, M. (2012). Using artifact-based interviews to study the development of computational thinking in interactive media design. Paper presented at annual American Educational Research Association meeting, Vancouver, BC, Canada.

Resources

PILA Module Framework

‘Computational problem solving’  is the iterative process of developing  computational solutions to problems.  This page will guide you through the conceptual framework that undelies the PILA CPS Module, exploring why CPS is a key skill for students to master and how PILA can help to assess and teach it in the classroom.

Computational Problem Solving Conceptual Framework

Conceptual framework of the PILA Computational Problem Solving module

Assessment Applications

Coming in 2022 - This module will assess students’ ability to understand and model how complex systems work.

Coming in early 2022 - Systems thinking is the process of building an understanding of the relationship between parts of a complex system, and how these relationships create certain dynamics within that system. It can help students create a deeper understanding of why things happen a certain way, and to construct a broader perspective that informs the type of questions they ask and solutions they imagine for complex issues. 

The digital tools and environments offered by PILA will provide students with tangible opportunities to examine the interdependencies and ever-changing variables that are characteristic of real-life phenomena, assessing their ability to understand and model how those work. 

Coming in 2022 - Systems Thinking is defined as the capacity to understand and model how complex systems work. Students will learn about scientific phenomena, and will be asked to explore and graphically represent the interaction among the different elements that characterize that phenomenon by creating concept maps.

Systems Thinking Module

In the PILA Computational Problem Solving module, students will work through dynamic problems in open-ended digital environments where they have to interpret, design or debug computer programs in a block-based format. To ensure students have the opportunity to learn and apply concepts and skills, the assessment experiences provide learning resources and other forms of support, such as worked examples, hints, automated feedback and tutorials.

The Computational Problem Solving framework defines the set of complementary computational problem solving skills and concepts, as well as learning skills assessed in this module. 

This module assesses students’ ability to develop structured, step-by-step solutions to problems.

Computational Problem Solving  and PILA

PILA_CPSFramework_June 2021.pdf

Computational problem solving is the iterative process of developing computational solutions to problems. These solutions are expressed as a logical sequence of steps (i.e. programs) for others to understand and for computers to execute. Developing such solutions allows individuals to create, organise, express and share ideas using digital tools. Given the complex and ill-structured problems students will increasingly face in their everyday lives, mastering computational problem solving skills is key for them to thrive in a rapidly changing 21st Century society.

Computational problem solving is the iterative process of developing computational solutions to problems. In this module, students work through dynamic problems in open-ended digital environments where they have to interpret, design or debug computer programs in a block-based format.

Computational Problem Solving Module

Welcome to PILA platform. This is a notification that only appears on the homepage.

This is a test notification on a single application page

This is the demo version of the PILA website!

Application Guides

Application Guides

Application Guides