Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions paths/chat/chat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ post:
partId: "part-1"
timeSpentOnPart: "10 minutes"
submission:
type: "TEXT"
format: "text:plain"
content:
text: 'outputs= ["Woof!", "Meow!"]'
value: 'outputs= ["Woof!", "Meow!"]'
feedback:
- feedbackId: "fb-101"
message: "Incomplete answer, explain why these outputs occur."
Expand Down
10 changes: 4 additions & 6 deletions paths/evaluate/evaluate-health.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,10 @@ get:
supportsSummativeFeedback: true
supportsDataPolicy: "PARTIAL"
supportedArtefactProfiles:
- type: "TEXT"
supportedFormats: ["plain", "markdown"]
- type: "CODE"
supportedFormats: ["python", "java", "javascript"]
- type: "MATH"
supportedFormats: ["latex", "mathml"]
- supportedFormats: ["text:plain", "text:markdown"]
- supportedFormats:
["code:python", "code:java", "code:javascript"]
- supportedFormats: ["math:latex", "math:mathml"]
supportedLanguages:
- "en"
- "de"
Expand Down
70 changes: 28 additions & 42 deletions paths/evaluate/evaluate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,9 @@
submission:
submissionId: "sub-123"
taskId: "task-42"
type: "TEXT"
format: "plain"
format: "text:plain"
content:
text: "Explain what polymorphism is in object-oriented programming."
value: "Explain what polymorphism is in object-oriented programming."
submittedAt: "2025-12-16T09:30:00Z"
version: 1
configuration: null
Expand All @@ -39,10 +38,9 @@
submission:
submissionId: "sub-777"
taskId: "task-42"
type: "TEXT"
format: "plain"
format: "text:plain"
content:
text: "My short answer..."
value: "My short answer..."
preSubmissionFeedback:
enabled: true
configuration:
Expand All @@ -56,10 +54,9 @@
submission:
submissionId: "sub-async-001"
taskId: "task-42"
type: "TEXT"
format: "plain"
format: "text:plain"
content:
text: "Detailed essay answer that may require longer processing."
value: "Detailed essay answer that may require longer processing."
submittedAt: "2025-12-16T09:45:00Z"
version: 1
callbackUrl: "https://learning-platform.example.com/hooks/evaluate-result"
Expand All @@ -76,7 +73,7 @@
- "Explain the concept of polymorphism."
- "Provide an example of subtype polymorphism in Java."
referenceSolution:
text: >
value: >
Polymorphism allows the same method call to result in different behavior
depending on the object's runtime type. For example, a variable of type
Shape can reference a Circle or Rectangle, and calling draw() will invoke
Expand All @@ -87,10 +84,9 @@
submission:
submissionId: "sub-456"
taskId: "task-12"
type: "TEXT"
format: "plain"
format: "text:plain"
content:
text: >
value: >
Polymorphism means that an object can take many forms, for example
subclasses implementing methods differently.
submittedAt: "2025-12-16T10:00:00Z"
Expand All @@ -105,11 +101,11 @@
- criterionId: "crit-1"
name: "Correctness"
context: "The explanation of polymorphism is conceptually correct."
maxPoints: 10

Check warning on line 104 in paths/evaluate/evaluate.yml

View workflow job for this annotation

GitHub Actions / Lint / Lint

no-invalid-media-type-examples

Example value must conform to the schema: `0` property must NOT have unevaluated properties `maxPoints`.
- criterionId: "crit-2"
name: "Clarity"
context: "The explanation is clear, well-structured, and easy to understand."
maxPoints: 5

Check warning on line 108 in paths/evaluate/evaluate.yml

View workflow job for this annotation

GitHub Actions / Lint / Lint

no-invalid-media-type-examples

Example value must conform to the schema: `1` property must NOT have unevaluated properties `maxPoints`.
preSubmissionFeedback:
enabled: false
configuration:
Expand All @@ -129,10 +125,9 @@
submission:
submissionId: "sub-code-001"
taskId: "task-python-101"
type: "CODE"
format: "python"
format: "code:python"
content:
code: |
value: |
def fibonacci(n):
if n <= 1:
return n
Expand All @@ -150,10 +145,9 @@
submission:
submissionId: "sub-code-002"
taskId: "task-java-201"
type: "CODE"
format: "java"
format: "code:java"
content:
files:
value:
- path: "src/Main.java"
content: |
public class Main {
Expand All @@ -179,10 +173,9 @@
submission:
submissionId: "sub-math-002"
taskId: "task-algebra-101"
type: "CODE"
format: "sympy"
format: "code:sympy"
content:
expression: "solve(x**2 - 4, x)"
value: "solve(x**2 - 4, x)"
imports:
["from sympy import symbols, solve", "x = symbols('x')"]
submittedAt: "2025-12-16T12:30:00Z"
Expand All @@ -194,10 +187,9 @@
submission:
submissionId: "sub-math-001"
taskId: "task-calculus-101"
type: "MATH"
format: "latex"
format: "math:latex"
content:
expression: "\\int_{0}^{\\infty} e^{-x^2} dx = \\frac{\\sqrt{\\pi}}{2}"
value: "\\int_{0}^{\\infty} e^{-x^2} dx = \\frac{\\sqrt{\\pi}}{2}"
submittedAt: "2025-12-16T12:00:00Z"
version: 1

Expand All @@ -207,10 +199,9 @@
submission:
submissionId: "sub-math-003"
taskId: "task-geometry-101"
type: "MATH"
format: "mathml"
format: "math:mathml"
content:
expression: |
value: |
<math xmlns="http://www.w3.org/1998/Math/MathML">
<mrow>
<mi>a</mi>
Expand All @@ -233,10 +224,9 @@
submission:
submissionId: "sub-model-001"
taskId: "task-oop-design-101"
type: "MODEL"
format: "uml"
format: "model:uml"
content:
model: |
value: |
@startuml
abstract class Animal {
+name: String
Expand All @@ -262,10 +252,9 @@
submission:
submissionId: "sub-model-002"
taskId: "task-database-101"
type: "MODEL"
format: "er"
format: "model:er"
content:
model:
value:
entities:
- name: "Student"
attributes:
Expand Down Expand Up @@ -299,10 +288,9 @@
submission:
submissionId: "sub-model-003"
taskId: "task-process-101"
type: "MODEL"
format: "bpmn"
format: "model:bpmn"
content:
model: |
value: |
<?xml version="1.0" encoding="UTF-8"?>
<bpmn:definitions xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL">
<bpmn:process id="OrderProcess">
Expand All @@ -323,10 +311,9 @@
submission:
submissionId: "sub-text-002"
taskId: "task-essay-101"
type: "TEXT"
format: "markdown"
format: "text:markdown"
content:
markdown: |
value: |
# Introduction to Polymorphism

Polymorphism is a fundamental concept in **object-oriented programming** that allows objects to be treated as instances of their parent class.
Expand Down Expand Up @@ -370,8 +357,7 @@
context: "The solution produces correct results for the specified problem."
maxPoints: 10
target:
artefactType: "TEXT"
format: "plain"
format: "text:plain"
locator:
type: "span"
startIndex: 0
Expand Down
2 changes: 1 addition & 1 deletion paths/evaluate/schemas/EvaluateCapabilities.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ properties:
- "null"
description: >
Optional list of supported artefact profiles. Each profile specifies
an artefact type and the formats supported for that type.
a set of supported formats.
items:
$ref: "../../shared/schemas/ArtefactProfile.yml"
supportedLanguages:
Expand Down
20 changes: 9 additions & 11 deletions paths/evaluate/schemas/FeedbackTarget.yml
Original file line number Diff line number Diff line change
@@ -1,24 +1,22 @@
type: object
description: Optional reference to a specific part of the submission.
required:
- artefactType
- format
properties:
artefactType:
$ref: "../../shared/enums/ArtefactType.yml"
format:
type:
- string
- "null"
type: string
pattern: "^(text|code|math|model|other):[a-z0-9_]+$"
description: >
Optional format specifier matching the submission format (e.g., python, latex).
Format of the targeted artefact, matching the submission's format
(see Submission.format).
locator:
type:
- object
- "null"
description: >
Optional locator into the submission content. Standard locator types:
- TEXT: { type: "span", startIndex: number, endIndex: number }
- CODE: { type: "range", file?: string, startLine: number, endLine: number, startColumn?: number, endColumn?: number }
- MATH: { type: "subexpression", path: string } (e.g., path to subexpression)
- MODEL: { type: "element", elementId: string, elementType?: string }
- text:*: { type: "span", startIndex: number, endIndex: number }
- code:*: { type: "range", file?: string, startLine: number, endLine: number, startColumn?: number, endColumn?: number }
- math:*: { type: "subexpression", path: string } (e.g., path to subexpression)
- model:*: { type: "element", elementId: string, elementType?: string }
additionalProperties: true
5 changes: 0 additions & 5 deletions paths/shared/enums/ArtefactType.yml

This file was deleted.

20 changes: 9 additions & 11 deletions paths/shared/schemas/ArtefactProfile.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,23 @@
type: object
description: >
Describes support for a specific artefact type and its formats.
Used in health/capabilities responses to advertise what a service can handle.
required:
- type
Describes support for a set of submission formats. Used in health/capabilities
responses to advertise what a service can handle.
properties:
type:
$ref: "../enums/ArtefactType.yml"
supportedFormats:
type:
- array
- "null"
description: >
List of supported formats for this artefact type. Use lowercase values.
If null or empty, the service accepts any format for this type.
List of supported formats, using the same namespaced "<category>:<format>"
convention as Submission.format (see Submission.yml). A category-wildcard
entry (e.g. "code:*") means the service accepts any format in that
category. If null or empty, the service accepts any format.
items:
type: string
examples:
- ["plain", "markdown"]
- ["python", "java", "javascript", "wolfram", "matlab"]
- ["latex", "mathml"]
- ["text:plain", "text:markdown"]
- ["code:python", "code:java", "code:javascript", "code:sympy"]
- ["math:latex", "math:mathml"]
contentSchema:
type:
- object
Expand Down
47 changes: 29 additions & 18 deletions paths/shared/schemas/Submission.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ description: >
A student's submission for a task. The structure of 'content' is intentionally
generic and task-specific.
required:
- type
- format
- content
properties:
submissionId:
Expand All @@ -14,29 +14,40 @@ properties:
- string
- "null"
description: Optional identifier of the task this submission belongs to.
type:
$ref: "../enums/ArtefactType.yml"
format:
type:
- string
- "null"
type: string
pattern: "^(text|code|math|model|other):[a-z0-9_]+$"
description: >
Optional format specifier providing additional detail about the artefact.
For TEXT: plain_text, markdown, html, rich_text, etc.
For CODE: programming language (e.g., python, java, javascript, wolfram, matlab).
For MATH: latex, mathml, sympy, wolfram, asciimath, etc.
For MODEL: uml, er, bpmn, petri_net, state_machine, etc.
Use lowercase values and snakecase. Services should document which formats they support.
Format of the submission content, as "<category>:<format>". The category
prefix determines how 'content' should be interpreted, and disambiguates
formats (e.g. "sympy" or "wolfram") that exist under more than one category.
Recommended values:
text:plain, text:markdown, text:html, text:rich_text;
code:python, code:java, code:javascript, code:matlab, code:sympy, code:wolfram;
math:latex, math:mathml, math:sympy, math:wolfram, math:asciimath;
model:uml, model:er, model:bpmn, model:petri_net, model:state_machine;
other:<custom> for anything not covered above.
Services should document which formats they support.
content:
type: object
required:
- value
additionalProperties: true
description: >
Logical representation of the submission content. The expected structure
depends on the artefact type:
- TEXT: { text: string } or { markdown: string }
- CODE: { code: string } or { files: [{ path: string, content: string }], entryPoint?: string }
- MATH: { expression: string }
- MODEL: { model: string | object, notation?: string }
Logical representation of the submission content. 'value' holds the
submitted content itself; its shape depends on 'format':
- text:*: value is a string
- code:* (single file): value is a string
- code:* (multiple files): value is [{ path: string, content: string }],
with an optional sibling 'entryPoint' naming the file to run
- math:*: value is a string (e.g. a LaTeX or MathML expression)
- model:*: value is a string or object, with optional sibling metadata
such as 'notation' or 'diagramType'
Format-specific auxiliary fields (e.g. 'entryPoint', 'imports', 'notation',
'diagramType') are additional sibling properties alongside 'value', not
nested inside it.
properties:
value: {}
submittedAt:
type:
- string
Expand Down
4 changes: 3 additions & 1 deletion paths/shared/schemas/Task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ properties:
type:
- object
- "null"
description: Optional reference or example solution (structure is task-specific).
description: >
Optional reference or example solution. By convention, follows the same
{ value: ..., <format-specific fields> } shape as Submission.content.
additionalProperties: true
metadata:
type:
Expand Down
Loading