Platform access level: Org Admins, Managers, and Writers (anyone who builds courses).
Questions are how a course proves it worked. A question can check that learners are able to apply what they learned, reveal whether they have already applied the behavior, get them to commit to applying it in a concrete scenario they're facing, or capture how they feel. Each of those answers is evidence of behavior change, and AI assessment turns even written answers into accuracy data you can report on.
Arist's question types live in two different places in the editor. The first is Add block, inside a lesson, where you add Multiple choice, Open-ended, and Rating questions alongside Media blocks for images and video. The second is Add content at the bottom of the editor outline, which is where the Confidence lift survey, NPS survey, and Learner sentiment question live. Those three are Course benchmarks, so they measure the course as a whole rather than one lesson.
This guide covers every question type, how to choose between them, how to set up AI assessment so open-ended questions grade themselves, and how to review the responses each type collects. It's for anyone who builds courses, whichever path the course started from. For the editing surface itself, including where each control sits and how lessons are written, see Using the Content Editor.
1. Adding a question block to a lesson
Every lesson-level question lives inside a lesson as a block. Open the course in the content editor, select the lesson, then select Add block to see your options.
2. Choosing the question type for the job
Each question type collects a different kind of answer, so pick based on what you want the response to tell you.
Question type | Where to add it | Learners answer with | Best for |
Multiple choice question | Add block, inside a lesson | One or more pre-selected options | Knowledge checks and scenarios with a clear right answer |
Open-ended question | Add block, inside a lesson | A written response, which AI can grade if you turn assessment on | Qualitative insight in learners' own words |
Rating question | Add block, inside a lesson | A numeric rating on a scale | Measuring confidence or gathering opinions |
Confidence lift survey | Add content > Course benchmarks | A confidence question at the start and at the end of the course | Proving the confidence shift the course created |
NPS survey | Add content > Course benchmarks | A likeliness-to-recommend question on the last lesson | Learning whether learners would recommend the course |
Learner sentiment | Add content > Course benchmarks | A satisfaction question on the last lesson | Measuring learner satisfaction with the course |
The Add block modal also offers Media, which isn't a question at all. It sends learners an image or video alongside the lesson.
Wherever you can, favor scenario-based application questions over pure recall. Asking a learner to apply an idea to a realistic situation tells you whether the training will hold up on the job, and it's the same approach Creator takes when it generates questions.
3. Setting up AI assessment on an open-ended question
AI assessment is optional, and you choose it per question. An open-ended question collects written replies for you to read either way, and one checkbox turns on AI grading.
To turn it on:
Add an Open-ended question block to the lesson.
Check This open-ended question has a correct answer. The block's header changes to Open-ended (AI evaluated).
Fill in the Criteria for a correct answer, describing what a correct answer includes.
Write the Response to learners who answer correctly and the Response to learners who answer incorrectly.
The AI compares each response against your criteria using semantic matching, so it scores meaning rather than exact wording, and it considers the lesson's context. Each response is graded correct or incorrect, the matching response message is sent to the learner, and the result counts toward the course's answer accuracy.
Grading is only as fair as the question, so ask questions that have a clear right and wrong answer. Some question shapes suit AI assessment better than others:
Good fits: Questions with an objectively correct answer, scenarios that allow several valid approaches but have clear wrong ones, and definitive knowledge checks all grade well.
Poor fits: Avoid creative or subjective prompts and opinion questions, because there's no correct answer to grade against.
The same thinking applies to the criteria you write. Describe the range of acceptable answers rather than one perfect one. Framing like "Some examples might include..." or "Key words a learner might mention include..." gives the AI clear criteria without demanding an exact match.
Tip: Balance your formats. Too many open-ended questions in a course lowers engagement, so mix them with multiple choice and keep each lesson to one or two questions.
4. Measuring the whole course with benchmarks
Course benchmarks are questions about the course itself rather than about one lesson's content. Open Add content at the bottom of the editor outline and pick from the Course benchmarks section; once a benchmark is in the course, the window shows it as already added.
Confidence lift survey: Adds a pair of questions, one at the start of the course and one at the end, measuring how confident learners feel. Comparing the two shows the immediate shift the training created, which makes it one of the simplest impact numbers you can report.
NPS survey: Adds a question to the last lesson measuring how likely learners are to recommend the course.
Learner sentiment: Adds a question to the last lesson measuring how satisfied learners are with the course.
Every course Creator builds begins and ends with a confidence-lift question automatically, and Creator also adds the Learner sentiment question to the last lesson for you. If you're building from scratch, add the benchmarks you want yourself; Arist places the confidence-lift pair on the first and last lessons for you.
5. Reviewing responses for every question type
Every question type reports back to the Analytics page, whichever formats you used. Filter the page to your course and open the Engagement tab. Click any question to read the exact responses learners gave, or select Response summary for an overview of every question's responses. What to look for depends on the type:
Multiple choice questions: These count toward the course's answer accuracy, and the response counts show you how learners split across the options.
Rating questions: Click into the question to read the ratings learners gave.
Open-ended questions: Read the replies in learners' own words; they explain the why behind your numbers. With AI assessment on, each reply also carries a grade, so you can see accuracy percentages and change a grade when you disagree with the AI's call.
Course benchmarks: Confidence lift, Learner NPS, and Learner sentiment each report as their own metric tile on the Analytics Insights tab.
Reviewing Analytics walks the whole Analytics page, from filters to tiles to the Engagement tab.
Related articles
Note: Need help at any point? Reach out to your Arist Customer Success contact, or email [email protected].