Creating a new task - INGInious Documentation

3.3.1 Tutorial

In this document we will describe how to create a simple task, that checks that a code in Python returns “Hello World!”.

Note: Demonstration tasks are made available for downloadhere. They can also be downloaded and installed automatically via theinginious-installscript.

Creating the task description Using the webapp

If you are using the webapp, this procedure can be done using the graphical interface:

1. Go to the Course administration/Tasks page, enter helloworld as a new task id and click on Create new task.

2. In the Basic settings tab, set the task name to Hello World! and put some context and author name. Con-tainer setup can be left with default parameters.

3. In the Subproblems tab, add a new code-type problem with problem id question1.

4. Set some problem name and context, and set language to python.

5. Save changes and go to Task files tab.

Manually

This is only possible if the administrator has given access to the course directory to the course administrator.

The task description is a YAML file describing everything that INGInious needs to know to verify the input of the student. Here is a simple task description. Put this file with the name task.yaml in a newly created helloworld folder in your course directory.

author: "The INGInious authors"

accessible: true name: "Hello World!"

context: "In this task, you will have to write a python script that displays 'Hello

˓→World!'."

3.3. Creating a new task 27

problems:

question1:

header: "def func():"

Most of the fields are self-explanatory. Some remarks:

• The field problems is a dictionary of problems. Each problem must have an unique id, for example here question1.

• Problem question1 have its type field that equals to code, which means the student must enter some code to answer the question. Other types exists, such as multiple-choice.

• The field limits are the limits that the task cannot exceed. The time is in seconds, and memory and output are in MB.

• The environment field is intended to change the environment where the tasks run. The available environ-ments are those you downloaded during installation or those you created by creating a grading container. Please seeCreating a new container image.

More documentation is available here:Creating a new task.

Creating the run file

In your task folder, you will put every file needed to test the input of the student. This folder content can be shown in the webapp in the Task files tab of the Edit task page.

1. Create a template file template.py, where we will put the code of the student.

def func():

@ @question1@@

func()

The syntax is very simple: put a first @ on the line where you want to put the code of the student. Then indent the line and write a second @. Now write the problem id of the problem you want to take the input from (question1) then write another @, write a possible suffix (not used here), and then finish the line with a last

2. Create the run file. This file will be the script that is launched when the task is started. Here we will create a bashscript, that parses the template and verifies its content.

#! /bin/bash

# This line parses the template and put the result in studentcode.py parsetemplate --output student/studentcode.py template.py

# Verify the output of the code...

output=$(run_student python student/studentcode.py) if [ "$output" = "Hello World!" ]; then

# The student succeeded

INGInious Documentation, Release 0.4

feedback-result success

feedback-msg -m "You solved this difficult task!"

else

# The student failed feedback-result failed

feedback-msg -m "Your output is $output"

Here we use three commands provided by INGInious, parsetemplate, run_student and feedback.

The code is self-explanatory; just notice the usage of run_student that ask INGInious (precisely the Docker agent) to start a new student container and run inside the command python studentcode.py.

Please note that the run_student command is fully configurable: you can change the environment on which you run the task, define new timeouts, memory limits, ... Seerun_studentfor more details.

3. If not using the webapp, don’t forget to give the run file the execution rights:

$ chmod +x helloworld/run More documentation is available here:Run file.

3.3.2 Task reference

Task description files

Inside a course folder (seeCreating a new course), tasks are identified by subdirectories named by their task id and containing a task.yaml file. For instance, this file, for a task with id taskid1, should be placed in a taskid1 subdirectory.

task.yamlis a YAML file containing information about the task.

author: Your name context:

|-The context of this task. Explain here what the students have to do.

order: 1 groups: false

problems:

a_problem_id:

• author, context, order, name, language and header are only needed if you use the frontend.

contextand header are parsed using restructuredText.

• order is an integer, used by the frontend to sort the task list. Task are sorted in increasing value of order.

• weight is a decimal value indicating the weight of the task score to use to compute the total course score.

3.3. Creating a new task 29

• accessible describes when the task is accessible to student. This field is not mandatory (by default, the task is visible) and can contain the following values:

true the task is always accessible false the task is never accessible

"START" where START is a valid date, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only acces-sible after START.

"/END" where END is a valid date, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible before END.

"START/END" where START and END are valid dates, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible between START and END.

• problems describes sub-problems of this task. This field is mandatory and must contain at least one problem.

Problem types are described in the following sectionProblem types. Each problem must have an id which is alphanumeric and unique.

• limits contains the limits that will be applied on the grading container. time is the CPU timeout in seconds, and hard_time is the timeout in real time.

By default, hard_time is defined to be to 3*‘‘time‘‘. This can leads to problems when INGInious is under heavy load, but allow to detect processes that do too much system interruptions (sleep calls or IO)

memoryis the maximum memory allowed to the container.

Please note that the limits of the student containers (container that you start inside the grading container) will use these limits by default.

• environment is the name of the Docker container in which the grading code will run. This field is only needed if there is code to correct; a multiple-choice question does not need it. This environment will be used by default for the student containers.

• groups allows to indicate if the submission is to be done individually or per groups/teams. (see Classrooms and Teams).

• network_grading indicates if the grading container should have access to the net. This is not the case by default.

• evaluate indicates the submission that must be used for evaluation. This can be either:

best This is the default value. In this case, the best submission is used.

last In this case, the last submission is used.

student In this case, the student can select the submission for evaluation. This allows student to select the submission for evaluation without submitting it again or if submission replays are planned. This feature is not available in the LTI module due to LTI specifications limitations, and will be considered as best submission.

• submission_limit indicates the amount of submissions a student can make within a certain period of time.

It is composed of two fields:

– amount is an integer value indicating the amount of submission. A value of -1 corresponds to an infinite amount of submissions.

– period is an integer value indicating the length of the submission period in hours. A value of -1 corre-sponds to an infinite period. At the end of this period, the student can submit amount submissions again during period hours.

• stored_submissions indicates the amount of submissions that must be saved in the submission history. A value of 0 keeps all the submissions.

INGInious Documentation, Release 0.4

Problem types Code problems

type: codeproblems allows students to submit their code. The code is then sent to a container where a script made by the teaching team corrects it.

Here is a simple example for a code problem type: code

language: c header:

|-Hello dear student!

I'm a multiline header!

header and language are only needed when using the frontend and are not mandatory. This description typically displays on the frontend a box where student can put their code.

optionalis an optional field, that defaults to false, that indicates if this problem is mandatory or not.

Code problem input’s are available in the run script (seeRun file) directly with the id of the problem.

Single code line problems

type: code-single-lineis simply a code box that allows a single line as input.

type: code-single-line language: c

header:

|-Hello dear student!

I'm another multiline header, parsed with *RST*!

Single line code problem input’s are available in the run script (seeRun file) directly with the id of the problem.

Advanced code problem

Advanced code problems are available:

type: code

header: some text

boxId1:

type: text

content: Some additional text boxId2:

3.3. Creating a new task 31

maxChars: 1000 lines: 8 language: java

Boxes are displayable (on the frontend) input fields that allows the student to fill more than one entry per prob-lem. Different box types are available, all of them are demonstrated above. Every configuration in the boxes (max-Chars,*lines*,*language*) is not mandatory, except content if the box type is text, and the field optional (default to false), that indicates if the box is mandatory or not.

In the run file (seeRun file), boxes input are available with the name problem_id/box_id

Match problems

Match problem are input that allows a single-line input from the student and that returns if the student entered exactly the text given in the “answer” field.

header: some text describing this problem answer: 42

Match problem input’s are available in the run script (seeRun file) directly with the id of the problem.

Multiple choice problems

header: The answer to life, the universe and any other things is multiple: true

limit: 2

error_message: "Wrong answer. Don't panic, and read Hitchhiker's Guide to the Galaxy."

success_message: "You're right! But don't forget to always take your towel with you."

choices:

feedback: "43 isn't the answer. Maybe can you try to substract one?"

- text: 41?

feedback: "41 isn't the answer. Maybe can you try to add one?"

Choices are described in the choices section of the YAML. Each choice must have a text field (on the frontend) that will be parsed in restructuredText. Valid choices must have a valid: truefield. The field feedback is a message that will be displayed when the student check the choice.

multipleindicates if the student may (or not) select more than one response.

Choices are chosen randomly in the list. If the limit field is set, the number of choices taken equals to the limit.

There is always a valid answer in the chosen choices.

error_messageand success_message are messages that will be displayed on error/success. They are parsed in RST and are not mandatory.

Multiple choice problem input’s are available in the run script (seeRun file) directly with the id of the problem. The input can be either an array of integer if multiple is true or an integer. Choices are numbered sequentially from 0.

INGInious Documentation, Release 0.4

Run file

When the student have submit his/her code, INGInious starts a new Docker container with the right environment for the task (as given in the .task file). Inside this container is launched a script, called run, that you have to provide in the directory of your task.

Here is a simple example of a run file, compatible with the default environment, that simply returns that the student’s code is OK:

#!/bin/bash

feedback-result success

The run script is simply an executable application (a bash script, a python script, or a compiled executable runnable by the container). INGInious’ default containers provides commands (also available as python libraries) to interact with the backend.

By default, the script is run inside the container in the /task directory, by a non-root user. You can modify the container to change this (and everything else).

Feedback commands

feedback (obsolete since v0.4)

The feedback command allows you to set the result of your tests. Every argument is optional.

-r, --result STATUS set the result to STATUS. STATUS can be - success (the student succeeded the task), - failed (there are error in the student answer), - timeout (the tests timed out), - overflow (there was a memory/disk overflow) or - crash (the tests crashed) -e, --escape interprets the space-like escape chars.

-f, --feedback MSG set the feedback message to MSG. It is possible to set different messages for each problems. You can use -i to change the problem to which you assign the message -i, --id PROBLEMID set the problem id to which the message from the -f option is assigned. Unused

if -f is not set.

-g, --grade GRADE the grade. Should be a floating point number between 0(no points) and 100(all points) (bonuses, up to 200, are allowed).

-c, --custom CUSTOM add a value VAL to the “custom” dictionnary, with key KEY, where CUSTOM is KEY:VAL, which is stored in DB. Useful for plugins.

-j, --custom-json CUSTOM same as `--custom` but VAL is a json-encoded dictionnary The feedback command can be called multiple times.

feedback --result success --feedback "You're right, the answer is 42!"

feedback-result

The feedback-result command sets the submission result of a task, or a problem, and uses the following syntax : feedback-result [-i|--id PROBLEM_ID] RESULT

The execution result can be of different types:

• success : the student succeeded the task

3.3. Creating a new task 33

• failed : there are error in the student answer

• timeout : the tests timed out

• overflow :there was a memory/disk overflow

• crash : the tests crashed

For instance, the following command will inform that the student succeeded:

feedback-result success

In Python : the equivalent command can be directly obtained with:

from inginious import feedback

feedback.set_global_result("success") # Set global result to success

feedback.set_problem_result("failed", "q1") # Set 'q1' subproblem result to failed

feedback-grade

The feedback-grade command sets the submission grade and uses the following syntax:

feedback-grade GRADE

If no grade is specified, the result score will be binary. This means that a failed submission will give a 0.0% score to the student, while a successful submission will give a 100.0% score to the student. For instance, the following command will give an 87.8% grade to the student:

feedback-grade 87.8

In Python : the equivalent command can be directly obtained with:

from inginious import feedback

feedback.set_grade(87.8) # Set the grade to 87.8%

feedback-msg

The feedback-msg command sets the feedback message associated to the task or a subproblem. It has several optional parameters:

-a, --append append to current feedback, if not specified, replace the current feedback.

-i, --id PROBLEM_ID problem id to which associate the feedback, leave empty for the whole task.

-e, --escape interprets backslash escapes -m, --message MESSAGE feedback message

If the message is not specified, the feedback message is read from stdin. For instance, the command can be used as follows:

feedback-msg -ae -m "This is the correct answer.\n\nWell done!"

In Python : the equivalent command can be directly obtained with:

INGInious Documentation, Release 0.4

from inginious import feedback

feedback.set_global_feedback("Well done !") # Set global feedback text to `Well done !

˓→`

feedback.set_problem_feedback("This is not correct.", "q1") # Set 'q1' problem

˓→feedback to `This is not correct.`

feedback-custom

The feedback-custom command sets a pair of key/value custom feedback, mainly used with plugins, and uses the following syntax :

feedback-custom [-j|--json] key value

The --json parameter indicates if value must be parsed as a JSON string. Please refer to the plugin documentation to know which value you have to set for key and value parameters.

For instance, the following command set the value 56 to the score key:

feedback-custom score 56

In Python : the equivalent command can be directly obtained with:

from inginious import feedback

feedback.set_custom_value("score", 56) # Set the `score` key to value 56

reStructuredText helper commands

Several helper commands are available to format the feedback text, which format is reStructuredText.

rst-code

The rst-code command generates a code-block with the specified code snippet and language to enable syntax high-lighting. It has the following parameters:

-l, --language LANGUAGE snippet language, leave empty to disable syntax highlighting -e, --escape interprets backslash escapes

-c, --code CODE snippet code

If the code parameter is not specified, it is read on standard input. The result is written on standard output. For instance, the command can be used as follows:

cat test.java | rst-code -l java | feedback-msg -a In Python : the equivalent command can be directly obtained with:

from inginious import rst

codeblock = rst.get_codeblock("java", "int a = 42;") # Java codeblock with `int a =

˓→42;` code

feedback.set_global_feedback(codeblock, True) # Appends the codeblock to the global

˓→feedback

3.3. Creating a new task 35

rst-image

The rst-image command generates a raw reStructuredText block containing the image to display. It has the following syntax

rst-image [-f|--format FORMAT] FILEPATH

The optional format parameter is used to specify the image format (jpg, png,...) if this is not explicitly specified the the image filename. The output is written on the standard output. For instance, the command can be used as follows:

rst-image generated.png | feedback -a

In Python : the equivalent command can be directly obtained with:

from inginious import rst

imgblock = rst.get_imageblock("smiley.png") # RST block with image

feedback.set_global_feedback(imgblock, True) # Appends the image block to the global

˓→feedback

rst-msgblock

The rst-msgblock command is used to generate a reStructuredText admonition in a specific colour according to the message type. It has the following optional parameters:

-c, --class CSS_CLASS Bootstrap alert CSS class:

• success

• info

• warning

• danger

-e, --escape interprets backslash escapes -t, --title TITLE message title

-m, --message MESSAGE message text

If the message parameter is not set, the message is read from standard input. For instance, the command can be used as follows:

rst-msgblock -c info -m "This is a note" | feedback -ae In Python : the equivalent command can be directly obtained with:

from inginious import rst

admonition = rst.get_admonition("success", "Yeah!", "Well done!") # RST message block

˓→of class "success" and title "Yeah!"

feedback.set_global_feedback(admonition, True) # Appends the block to the global

˓→feedback

rst-indent

The rst-indent command is used to handle the indentation of the given text. It has the following optional arguments:

-e, --escape interprets backslash escapes

INGInious Documentation, Release 0.4

-c, --indent-char INDENT_CHAR indentation char, default = tabulation -a, --amount AMOUNT amount of indentation, default = 1

-m, --message MESSAGE message text

If the message parameter is not set, the text is read from standard input. The amount of indentation can be negative to

In document INGInious Documentation (Page 31-47)