Assessment.md

Assessments Module

V1.0

Last Updated: 9th Sept 2022

Glossary/ Acronyms

CB	Capacity building
FRAC	Framework of Roles, Activities and Competencies
iGOT	Integrated Government Online Training
MCQ	Multiple choice questions
PIAA	Proctored independent, authorised assessment
PRD	Product requirement document
QR	Questions repository
QR_C	Questions repository contributor
QR_M	Questions repository manager
SOP	Standard operating procedure
UPTSU	Uttar Pradesh technical support unit

1. Overall Purpose and Vision

The scope of the document is to define the purpose and functionality of the knowledge based assessment tool for assessing the competency proficiency levels of individual candidates, specifically in knowledge. It lays out detailed users, functional requirements with flows. It also defines the desired outputs of each functionality. The document also details out how the self practice assessment and proctored assessment modules will interact and communicate with other digital systems. The assessment is part of PIAA (proctored, independent, authorised assessment) which is used for identifying the competency proficiency levels of candidates for all competencies that the participants require. 


The document is designed by the UPTSU team for technical partners, and vendors to clearly understand the requirements of the knowledge based assessment tool. 

2. **Project scope **

The module will enable creation and conduction of written and objective competency linked assessments in two setups:

1. self practice assessment for the candidate to practice and gauge whether they have attained the competencies mapped to their roles or position
2. Proactored, independent, authorised assessment (PIAA) in a designated PIAA centre to maintain credible competency attainment records for candidates and to provide inputs to the verifiable resume / competency passbook of each individual

Assumptions and dependencies

1. A competency directory, mapped with each position, role and activity is ready to refer for Assessment composition
2. Databases like list of PIAA centres ; PIAA setter; mapping of assessment process owners and other system users is available when setup starts
3. Overview of use case and interface requirements

The MCQ based assessment tool will have 6 roles namely;

#	Entity	Roles
1	Faculty	Question repository contributor (QR_C)
2	UPTSU	Question repository manager (QR_M)
3	PIAA setter
4	PIAA nodal
5	Candidate
6	Admin

Noun verbs

1. Questions - Create (QR C), Review (QR M)
2. Blueprint for Question paper - Create (PIAA setter)
3. PIAA papers - Create (PIAA setter), Review (PIAA Nodal)
4. Practice papers - Create (candidate), Submit (candidate)
5. PIAA paper submissions - Create (candidate), Update (PIAA Nodal), Review (?), View (?)
6. Verifiable credentials - Create (?), Issue (?), View (candidate)

Use case overview

Based on the requirements, some of the use cases for users are listed below.

S.No	Use Case	Description	User Role
1	Proposes competency wise questions to questions repository	As a QR contributor, I am able to record and propose various types of questions in the questions repository linked to competency levels.	Question Repository Contributor
2	Review questions: Edit, approve or reject questions	As a QR manager, I can view proposed questions, make edits and reject or accept the questions proposed. I can also edit the mapping of the questions to the mapped competency level.	Question Repository Manager
3	Manages and maintains question repository	As a QR manager, I am able to approve previously rejected questions or disapprove previously approved questions	Question Repository Manager
4	Compose assessment blueprints for various roles and cadres in public health	As a PIAA setter, I am able to select relevant competencies from the directory and set the assessments blueprint for various roles.	PIAA Setter
5	Initiate assessment session for candidates and facilitate	As a PIAA Nodal, I am able to set up assessment blueprints on systems for different candidates and begin / pause / stop assessments for them	PIAA Nodal
6	Take a practice self practice assessment	As a candidate, I am able to select any of the available question papers from the directory and conduct a self practice assessment and submit my responses in a timed manner. I am also able to get the results and feedback.	candidate
7	View and download certificate	As a candidate, I am able to view and download certificates for the PIAA assessments I have taken so far	candidate
8	Review assessments, approving / aborting assessment blueprints for the evaluation of results	As an admin, I am able to review assessment sessions and approve or abort them to decide if they can be sent for evaluation or not	Admin
9	View results and issue certificates to assessment takers	As an admin, I am able to view results and issue certificates to the assessment takers	Admin

Versioning

Use case #	Description	Version
1	Proposes competency wise questions to questions repository Question type: Single select Question type: Multiple choice questions – Single/Multi MCQ type , One-word answers, Fill in the blanks, Short/long answers, Case study-based questions, Match the following	v1 v2
2	Review questions: Edit, Approve or Reject questions	v1
3	Manages and maintains questions repository	v1
4	Compose assessment blueprints for various roles and cadres in public health of questions by competency levels # of questions by question type	v1 v2
5	Initiate assessment session for candidates and facilitate Individual - PIAA nodal opens the assessment blueprint on a desktop computer and maps it to a candidate and starts the assessment Batch start	v1 v2
6	Take a self practice assessment Untimed Timed	v1 v2
7	View and download certificate	v1
8	Review assessments, approving / aborting assessment blueprints for the evaluation of results	v1
9	View results and issue certificates to assessment takers PDF generator enabled certificate Verifiable credentials	v1 v2

Use case details

Use case 1: Proposes competency wise questions to questions repository

1. QR contributors logs in with Unique user id and password
2. The home page will include two tabs namely ‘Create questions’ and ‘View questions’
3. The home page will display the following metrics
4. Number of questions created
5. Number of questions approved
6. Number of questions rejected
7. Number of questions pending review
8. When QR contributors click on ‘Create questions’, a dropdown of position and competency directory will be shown to select competency level (Position-Competency-Competency level). Once the competency level is selected, a window to record the question body, options and correct answer input will be provided. Question body may contain pure text, text and image or only image also. This can be the method for adding one question at a time. For the v1, only single-select MCQ type questions can be added.
9. For a set of questions, a QR contributor is able to upload a set of questions in the repository in a specific format (csv) defined by system admin by clicking on ‘Upload questions’ CTA in the ‘Create questions’ page
10. The QR_C can see the questions created in the ‘View questions’ tab
11. On adding questions (single or bulk upload), the QR_C gets options to either ‘Save’ or ‘Submit for Review’
12. The questions will be displayed along with any one of the following status
13. Approved
14. Approved with edits
15. Sent for review
16. Pending review
17. Rejected
18. The questions will also have CTAs to send for review, edit or delete the questions
19. The user will be able to edit and delete only those questions which have their status as “Pending review”
20. The QR_C cannot resubmit questions which have been rejected
21. QR_C can click on the ‘Send for review’ CTA against a question to send it for review. QR_C can also multi-select the questions and submit them for review.

Types of Assessment questions (v2) :

The written assessments for competencies may use various types of items. A pop-up will be displayed while the QR_C is creating questions where they can select the type of questions. These types of items in the assessment item bank can be:

1. Multiple choice questions – Single/Multi select: using text/ images
2. One-word answers, Fill in the blanks
3. Short/long answers
4. Case study-based questions
5. Match the following

The selection of a particular/ mix of question types for an assessment blueprint is a choice of the PIAA setter. The question repository will have all types of questions mapped to each competency and competency level. For the first two types of questions ( MCQs and one word/ fill in the blanks) the mapping to a particular competency and competency level will be 1:1. For the others, it might be 1:many

Use case 2: Review questions: Edit, Approve or Reject questions

1. QR manager logs in with Unique user ID and password. On successful login, the user would be able to see the home page with the following tabs; ‘Review questions’ and ‘Manage reviewed questions’
2. The home page will display the following metrics
3. Number of approved questions
4. Number of questions pending review
5. Number of rejected questions
6. Once the QR_M clicks on ‘Review questions’, all the questions appear for review. This list will have multiple options to filter/sort based on position, roles, competency and competency levels, and question type.
7. If QR_M wants to edit the question, a button will be available next to each question. On clicking this, a new window will open with the option to edit the body of the question, the body of the alternatives and the mapping of the question to competency level. On clicking ‘Save and Update’, question will be tagged as “approved with edits”

Use case 3: Manages and maintains questions repository

1. On clicking on the ‘Manage reviewed questions’ tab, the list of questions which have been approved and rejected by the specific QR_M will be displayed
2. QR_M will have the option to multi-select questions and edit their status - i.e., previously approved questions can be rejected and vice versa

S. No	Functionality	User
1	Propose competency wise questions to questions repository	QR contributor
2	Review questions : Edit, Approve or Reject questions	QR Manager
3	Manage and maintain question repository	QR Manager

Use case 4: Compose assessment blueprints for various roles and cadres in public health

1. PIAA setter logs in using unique user id based login and password
2. The PIAA setter creates an assessment blueprint by tagging target audience (positions)
3. The PIAA setter adds attributes and tags to the assessment blueprint
   1. Name of assessment
   2. Description
   3. Mode - practice question papers vs PIAA only vs both
4. Further, the PIAA setter adds additional tags to the assessment blueprint by selecting a set of competencies (dropdown) from the competency directory. The competency directory will only reflect those competencies which are applicable to the position selected by the PIAA Setter in the previous step
5. The platform will list all the questions applicable to the competencies of a position
6. The PIAA Setter will have an option to multi-select the questions and lock them for PIAA. These questions will not display in the practice question papers for candidates
7. The PIAA setter will define the assessments blueprint: 4. Blueprint will include - # of questions by competency levels (version 2 to also include # of questions by question type) 5. System will pick randomly during conduction of the assessment from locked questions only as per the defined blueprint
8. Preview and finalise the assessment blueprints and has the option to make them go-live
9. Once an assessment blueprint is finalised and made live, it will be visible to the relevant positions for self practice assessment and to the PIAA Nodal for those positions
10. Homepage will have all assessment blueprints – draft-in-process, finalised but not live and currently live. PIAA setters will be able to remove assessment blueprints from live, edit assessment blueprints and delete existing blueprints at any stage.

Use case 5: Initiate assessment session for candidates and facilitate

1. PIAA nodal logins in with unique ID and password based login - Authentication of candidate will be handled offline.
2. The PIAA nodal is able to see all live and PIAA tagged assessment blueprints on the home screen with positions and competency levels tagged for each blueprint
3. The same login should work simultaneously such that the PIAA nodal can be logged in with all desktop computers at the PIAA centre
4. The assessment blueprint can be started as follows:
   1. Individual - PIAA nodal opens the assessment blueprint on a desktop computer and maps it to a candidate and starts the assessment
   2. Batch start can be part of Version 2
5. Once the assessment goes live, the assessment taker is shown an instructions page and the timer for the assessment starts once the assessment taker presses ‘Okay’
6. The assessment should allow moving from one question to another, to edit responses and to move back to earlier questions to review / update answers.
7. The PIAA nodal can pause, resume, stop assessment for any assessment taker on the system
8. The PIAA nodal should be able to submit a feedback / issues form after every assessment to be able to submit the data to the admin
9. The assessment taker can provide feedback for the assessment sessions (This would be optional. After completion of each assessment, a mail with the feedback form would be sent where the feedback can be raised)

Use case 6: Take a self practice assessment

1. candidate logs in with unique ID and password
2. Once successfully logged in, the candidate has the following tabs/sections on the home page – User profile (editable data fields for the individual. Mandatory on first login) ‘self practice assessment’ and ‘View Certificate’
3. By clicking on the self practice assessment tab, the list of available assessment blueprints for that position are displayed, by clicking on the blueprint, the competency lists being assessed in the blueprint can be viewed.
4. The candidate selects ‘take the self practice assessment’ for any of the available blueprints and starts the self practice assessment after an instructions screen.
5. Each time the self practice assessment is triggered for any assessment blueprint, based on the pre-defined composition algorithm, random questions are selected from the question bank and displayed in a random order to the candidate. The questions locked in PIAA assessment blueprints won’t be displayed in self practice assessment
6. An option for timed or untimed assessment may be given to the candidate (version 1 can be untimed with version 2 having both options)
7. candidate submits their responses to all the questions in the self practice assessment and submits. Option to review answers, move back and forward on the questions and end test without answering all questions may be provided.
8. candidates are shown the score achieved and have the option to scroll through the questions with the answer given by them and the correct answer indicated against each question.

Use Case 7: View and download certificate

1. Candidate logs in with unique ID and password
2. Once successfully logged in, the candidate has the following tabs/sections on the home page – User profile (editable data fields for the individual. Mandatory on first login) ‘self practice assessment’ and ‘View Certificate’
3. By clicking on the view certificate tab, the candidate will be able to see a list view of the certificates for the assessments they’ve taken so far

#	Assessment Code	Date taken	Status of issuance
1	GNM064	01/09/2022	VIEW CERTIFICATE (download icon)
2	GNM098	09/09/2022	In process

4. By clicking on the ‘View Certificate’ CTA, they will be able to view their certificate
5. Additionally, there would be an option to download the certificate through a download icon across the particular assessment taken

S. No	Functionality	User
4	Compose assessment blueprints	PIAA setter
5	Initiates a PIAA session	PIAA Nodal
6	Generate and initiate a practice question paper	candidate
7	View and download certificate	candidate

Use case 8: Review assessments, approving / aborting assessment blueprints for the evaluation of results

1. Admin logs in with unique ID and password
2. Admin’s home page will have the following tabs: ‘Review assessment sessions’ and ‘Issue certificates’
3. The Admin selects the ‘Review assessment sessions’ tab
4. Admin reviews all assessment sessions from issues / comments shared by the PIAA nodal and takes a decision to approve/ abort an assessment session based on the SOPs. Only approved assessments will be sent for calculation of results by the UPTSU

Use Case 9: View results and issue certificates to assessment takers

1. Admin logs in with unique ID and password
2. Admin’s home page will have the following tabs: ‘Review assessment sessions’ and ‘Issue certificates’
3. The Admin selects the ‘Issue certificates’ tab
4. This tab will reflect a list view of candidates whose results have been evaluated and generated by UPTSU along with the status of certificate issuance

#	Name of the candidate/ Assessment Taker	Assessment Code	Status of Issuance
1	Sagari	GNM063	Certificate issued
2	Rinkle	ANM023	ISSUE CERTIFICATE

5. The admin can find a CTA to issue certificates across the name of each candidate. By clicking the same, a certificate will be generated and issued to the candidate
6. The admin will also have an option to multi select/ select all candidates and issue them the certificates upon which the certificate will be issued to the candidates

S. No	Functionality	User
8	Review assessments, approving / aborting assessment blueprints for the evaluation of results	Admin
9	View results and issue certificates to assessment takers	Admin