Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Introduction

The Evaluation module is a tool designed to automate the crew assessment process onboard. The module provides you with a customizable workflow that enables team or department leaders, who act as evaluators, to communicate and share evaluation forms. With this customizable workflow, the system captures evaluations from evaluators and an Evaluee. This feature allows approvers and shore-based supervisors to compare the evaluator's assessment with the crew member's self-assessment.

Workflow

The Evaluation module allows circulating evaluations between seafarers based on the predefined chain of approval. The involved participants are then represented as:

...

The evaluation workflow outlines different stages that an evaluation document undergoes during the assessment process. For instance, an evaluation might undergo multiple rounds of assessments before the Evaluator gives their approval. Once approved, the evaluation process progresses to the subsequent level in the Chain of Approval (CoA). The crew member's position determines the choice of appraisal template. Consequently, a single CoA can support multiple workflows.

Module Peculiarities

Evaluation Form Template

An evaluation form is based on a template containing one or more sections. A section describes the evaluation topic and exists out of several questions. The answer type defines the format of the question and can be a free text field, multiple choice, or a set of values.
The evaluation form is divided into two: the left contains the crew members' assessment, while the evaluator uses the right.  Whether the evaluator and or Evaluee section is available is determined by the workflow stage.

Weighing Result

You can determine the evaluation form's score by considering the importance of answers, questions, and sections. Once done, the system calculates the weighted score for each question, which involves multiplying the actual percentage of the score by the question's weight. Then, it aggregates these question scores to calculate the overall score for the section. Similarly, the system computes the weighted score for each section, and by summing the scores for all sections, we arrive at the overall calculated score for the Evaluation form.

Expand
titleCalculation Rules and Examples

Qw=Question Weight

Qsw = sum of the weight of all questions in the section

Qs= Question score (Defined by the answer type)

Qcw= Question calculated weight= 100/Qsw* Qw

Qcs  =Question calculated score=   (Qcw/100) *Qs

 

  • Qw (Question Weight): It represents the weight assigned to a particular question. This weight indicates the relative importance or significance of the question within its section.

  • Qsw (Sum of Question Weights): It refers to the sum of the weights of all questions present in a specific section. It provides the total weight assigned to all the questions in that section.

  • Qs (Question score): It represents the score associated with a particular question. This value reflects the response or assessment given to the question.

  • Qsc (Question calculated score): This variable calculates the real value or adjusted score for a question. It is calculated using the formula: (100/Qsw) * Qw * Qv. It considers the question weight, question value, and the total weight of questions in the section.

Info

Example:
A section exist out 5 questions Q1, Q2, Q3,Q4, Q5. They are weighted as follows

Q1 :Qw=10, Qs=5
Q2: Qw=20, Qs=5
Q3;Qw=5, Qs=1
Q4:Qw=25,Qs=10
Q5:Qw=10, Qs=7
The following 4 steps calculate the overall score for the section:

Step 1: Calculate Qsw (Sum of Question Weights)
Qsw = QWq1 + QWq2 + Qwq3 + Qwq4 + Qwq5
Qsw = 10 + 20 + 5 + 25 + 10
Qsw = 70

Step 2: Calculate Qcw (Question Calculated Weight) for Each Question
The formula for calculating the Question Calculated Weight (Qcw) is:
Qcw = 100/Qsw x Qw

Q1
QcwQ1 = 100/70 × 10
QcwQ1 =1000/70
QcwQ1 ~ 14.29

Q2
QcwQ2 = 100/70 × 20
QcwQ2 =2000/70
QcwQ2 ~ 28.57

Q3
QcwQ3 = 100/70 × 5
QcwQ3 =500/70
QcwQ3 ~ 7.14

Q4
QcwQ4 = 100/70 × 25
QcwQ4 =2500/70
QcwQ4 ~ 35.71

Q5
QcwQ5 = 100/70 × 10
QcwQ5 =1000/70
QcwQ5 ~ 14.29

Step 3: Calculate Qcs (Question Calculated Score) for Each Question
The formula for calculating the Question Calculated Score (Qcs) is:
Qcs = Qcw/100 xQs

Q1.
QcsQ1 = 14.29/100 × 5
QcsQ1 =0.7143 x5
QcsQ1 ~ 3.57

Q2.
QcsQ2 = 28.57/100 × 5
QcsQ2 =0.2857 x5
QcsQ2 ~ 1.43

Q3.
QcsQ3 = 7.14/100 × 1
QcsQ3 =0.0714 x1
QcsQ3 ~ 0.07

Q4.
QcsQ4 = 35.71/100 × 10
QcsQ4 = 0.3571 x10
QcsQ4 ~ 3.57

 Q5.
QcsQ5 = 14.29/100 × 7
QcsQ5 =0.1429 x7
QcsQ5 ~ 1.00

 Step 4: Calculate the Overall Score for the Section
The overall score for the section is the sum of all the Question Calculated Scores (Ocs).
Overall Section Score = Qcsq1 + Qcsq2 + Qcsq3+ Qcsq4 + Qcsq5
Overall Section Score = 3.57 + 1.43 + 0.07 + 3.57 + 1.00
Overall Section Score ~ 9.64
So the overall calculated score for the evaluation form would be approximately 9.64

Ss= Section score is the sum of Qcs

Sw=Section Weight

Ssw=sum of weights of all sections in the template

Scw= Section calculated weight= (100/Ssw)*Sw

Scs =Section calculated score (Scw/100) *Ss

 

  • Ss (Section Score): It represents the cumulative value or score for a section. It is the sum of the Qsc values for all questions within that section.

  • Sw (Section Weight): It refers to the weight assigned to a particular section. Similar to question weight, it indicates the relative importance or significance of the section within the template.

  • Ssw (Sum of Section Weights): It represents the sum of the weights of all sections in the template. It provides the total weight assigned to all sections.

  • Scw (Section Real Value): This variable calculates the real value or adjusted score for a section. It is calculated using the formula: ((100/Ssw) * Sw) * Sv. It considers the section weight, section value (sum of Qrv values), and the total weight of all sections in the template.

  • Scs (Section calculated score)

  • Template Value: It represents the overall value or score for the entire template. It is the sum of the Scs values for all sections within the template.

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Similarly we are calculating the section scores
Example

Section weighted
S1 : Sw =25, Sc =10
S2 : Sw =50, Sc =75

Step 1: Calculate Section (Sum of Section Weights)

Ssw = 25+50 = 75

Step 2: Calculate Scw (Section Calculated Weight) for Each Question

Scw = (100/Ssw) x Sw

S1.
Scws1 = (100/75)*25=33.33

S2.
Scws2 = (100/75)*50=33.33

Step 3: Calculate Scs (Section Calculated Score) for Each Question

Scs= (Scw/100)*Sc

S1.

Scss1 = 33.33/100 × 10
Scss1 = 3.3

S2.

Scss2 = 66.66/100 × 75
Scss2 = 49.99

Step 4: Calculate the Overall Score for the Form

Overall Form Score = Scss1 +Scss2

Overall Form Score = 3.3+49.99 = 53.29

Overall Form Score ~ 9.64

Functionality

This section contains a description of the Evaluation features. By adhering to these guidelines, you will be on the right path to effectively utilizing the module.

...

  • My Evaluation (available to Coordinators and Subordinates). The tile displays a crew member’s ongoing and processed evaluations. For more information, see Evaluee Workflow.

  • Co-Workers Evaluations (available to Coordinators and Approvers). The tile allows the creation and management of evaluation forms. For more information, see Evaluator Workflow.

Anchor
Evaluator_Workflow
Evaluator_Workflow
Evaluator Workflow

The Co-workers Evaluation tile is only available for Evaluators predefined in the Chain of Approval. For more information, see Define Chain of Approval.

...

  • Send evaluation forms to Evaluees and schedule meetings to discuss details. For more information, see Send Evaluation Form.

  • Assess the Evaluee's skills based on the evaluation forms filled out by an Evaluee. For more information, see Execute Evaluation.

  • Check the status of evaluation forms sent to the Approver. For more information, see Check Evaluation Status.

Anchor
Send_Evaluation_Form
Send_Evaluation_Form
Send Evaluation Form

As an Evaluator, you can send out the evaluation forms to . For this, on the toolbar, click Create Evaluation:

...

As a result, the evaluation form is sent to the Evaluee. The evaluation form status is set to Created:

...

Anchor
Execute_Evaluation
Execute_Evaluation
Execute Evaluation

On sending the evaluation form to a crew member, as an Evaluator, you are waiting until the evaluation form is filled out and sent back to you for review. To review and process the evaluation form, navigate to Evaluation > Co-workers Evaluation > Execute Evaluation:

...

  • REJECT sends the evaluation form back to a teammate. The evaluation form moves back to the previous stage.

  • CANCEL discards the changes made in the form.

  • APPROVE sends the evaluation form to an approver for further processing.

Check List of Co-Workers

Switch to the List of Co-Workers page listing your teammates based on the predefined chain of approval. For more information, see Define Chain of Approval.

...

In the list, you can check for whom you can create or execute evaluation forms.

Anchor
Check_Evaluation_Status
Check_Evaluation_Status
Check Evaluation Status

As an Evaluator (coordinator), you can check the evaluation form status. For this, switch to the Status of Evaluation page that shows your teammates' evaluation forms and their current status:

...

As an approver, you can choose to view, approve, or reject the evaluation form by clicking the respective command.

Anchor
Evaluee_Workflow
Evaluee_Workflow
Evaluee Workflow

The My Evaluation tile is intended for Evaluees required to pass an evaluation of their skills and/or performance.

...

  1. Navigate to Evaluation > My Evaluation to view evaluation form(s) pending for processing:

...

Process Evaluation

On accessing the evaluation form, you can now proceed to fill it out. For this, first, make sure you are on the My Evaluation page displaying evaluation form thumbnails :

...

  • SAVE stores changes made.

  • SUBMIT saves applied changes and sends a form back to an Evaluator.

  • CANCEL discards changes made in the form.

Housekeeping

This section enables you to perform a one-time setup of the module both in the Adonis Administrator Portal and Adonis Crew Portal.

Adonis Crew Portal Setup

To get started, log in to the APP Crew Portal. When the Home page appears, click the Evaluation tile:

...

On this page, click the Housekeeping tile to start configuring the module:

...

Anchor
Set_up_Evaluation_Workflow
Set_up_Evaluation_Workflow
Set up Evaluation Workflow

On defining the chain of approval in the Adonis Administrator Portal (see Define Chain of Approval), you need to separately define the workflow between a coordinator, a subordinate, and an approver. For this, navigate to the Evaluation module > Housekeeping > Setup Workflow:

...

Info

You can reorder the created stages by dragging and dropping them. Reordering applies to all the stages except for Created and Approved.

Create Template

You can create different templates for electronic forms intended for evaluating crew members' performance.

...

On adding all the template components, refer to Compose Template to get to know how to create a template.

Anchor
Add_Form_Answers
Add_Form_Answers
Add Form Answers

To add form answers, first, navigate to Evaluation module > Housekeeping > Template. The Answer tab then opens up:

...

As a result, a new record becomes available on the Answer tab.

Anchor
Add_Form_Questions
Add_Form_Questions
Add Form Questions

To add form questions, first, navigate to Evaluation module > Housekeeping > Template > the Question tab:

...

When completed, click SAVE. As a result, a new record becomes available on the Question tab.

Anchor
Add_Form_Sections
Add_Form_Sections
Add Form Sections

To add form sections, first, navigate to Evaluation module > Housekeeping > Template > the Section tab:

...

When done, click SAVE to add a new form section to the table.

Anchor
Compose_Template
Compose_Template
Compose Template

To create a form template, first, navigate to Evaluation module > Housekeeping > Template > the Template tab:

...

When done, click SAVE to add a new template to the table.

Anchor
Set_Evaluation_Rule
Set_Evaluation_Rule
Compile Evaluation Form

After defining various workflows and composing templates, you can now specify a workflow and a template based on which crew members are to be evaluated. For this, navigate to Evaluation module > Housekeeping > Selection Criteria:

...

When completed, click SAVE to add a new evaluation rule to the table:

...

Adonis Administrator Portal Setup

This section will walk you through the setup required in the Adonis Administrator Portal.

Anchor
Define_Chain_of_Approval
Define_Chain_of_Approval
Define Chain of Approval

To get to know how to organize the hierarchical dependency of the approval chain, see Define Chain for Vessel.

...