Page tree
Skip to end of metadata
Go to start of metadata

The question type programming interaction is a special input interaction. If participants enter program code, this code will be evaluated either by defined test cases or a template with a corresponding return value. The automatic assessment is done using an interface for the server application  VPL. Supported programming languages include C, C++, Haskell, Java, Mathlab/Octave and Phython.

Example Programming interaction


A licence is required to use this question type.

Content

Question

FunctionDescription

Question

Question

Questions come with a WYSIWYG editor. This allows you to design your questions to your liking and include media and formulas.

Score

Score

By default, the maximum score for the question will be displayed. To change this setting, switch to the Scoring tab.

Answers

Answering a programming interaction is done by entering program code. To do this, the participant has access to a code editor with syntax highlighting. The author must choose the desired programming language and can enable an automatic assessment.

FunctionDescription

Auto Assessment

Auto assessment

By enabling this option, you allow the programming interaction to be automatically assessed. You can choose whether the assessment should be done using Test cases or the Return value of a template. If automatic assessment is not enabled, the question will be manually assessed, and the participant will receive a corresponding notification during the test or in the results view.

Programming language

Programming language

Select the expected programming language for entering and evaluating the question. Supported programming languages include C, C++, Haskell, Java, Mathlab/Octave and Phython.

Scoring

Scoring

If automatic assessment is enabled, you can choose from two scoring options:

  • Test cases: The scoring will be based on the number of correct test cases. The participant receives percentages of the maximum score per correct test case.
  • Return value of template: The scoring will be performed using a template, with the program code returning a percentage value as console output ("Grade: = >> 100").

Test cases

Test cases

If the option Test cases has been selected for scoring, a corresponding input field will be displayed. Test cases must be entered in VPL syntax. Each test case must be described as follows:

  • Case = case name
    Defines the start of a (new) test case. The case name can be used in the results report to see which test cases were successful and which failed.
  • Input = text (optional)
    Defines the input for the program code. The input can span several lines and ends with the next VPL expression (e. g. output).
  • Output = text
    Defines the expected output for the program code. The output can span several lines and ends with the next VPL expression (e. g. output).

Any number of test cases can be defined. Scoring is done according to the number of correct test cases. For the program code written by the participant, it is checked whether the defined output will be reached using the corresponding input. The number of successful test cases in relation to the total number of test cases will then result in the score, calculated as a percentage of the maximum score defined by the author. 

The VPL documentation gives an overview of the VPL syntax and shows more examples in the section Creating automated tests.

Template

Template

Regardless of the selected scoring option, the author may define program code as a template in which the program code entered by the participant will be integrated. The placeholder {LEARNERRESPONSE} defines where exactly the participant input will be inserted. If no template has been defined, the program code of the participant will be executed on its own.

The template can be used in addition to scoring. If the scoring option "return value of template" has been configured, the program code must return a percentage value as a console output in the following format:

  • Grade: =>> 100
    Scoring (grade) is indicated as a percentage. The score will be calculated as a percentage of the maximum score defined by the author.

Solution

Solution

The hint at the solution will be displayed to the participant as a sample solution, e.g. while they are running a test for which solutions have been enabled, or in the results view after completing a test.

Pre-defined content

Pre-defined content

If you enter pre-defined content, the input field in test run will be pre-filled with that text. The user may change this content.

More information

Code editor

For entering the program code, both the participant and the author have access to a special code editor based on the ACE editor (https://ace.c9.io/). In particular, this enables an improved overview due to syntax highlighting, specific indentation and outdentation, and program code-specific operator support. The editor will be preconfigured according to the selected programming language.

Code editor

Results report (VPL output)

The results data contains the full VPL output as a text file. This file will be stored in the data archive for each test run and participant. In the future, testers may look forward to a direct output display in the results views.

Results report