/
User Importer (as Integrator) - V4.8

User Importer (as Integrator) - V4.8

Owned by Stefan Spiegel
Last updated: Aug 23, 2024
9 min read

User Importer Configuration

Registry

A registry entry at HKEY_LOCAL_MACHINE\SOFTWARE\MarineLS\Integrators\[NameOfIntegrator] (e.g. NIUserCsvIntegrator) must be created and a new String Value added with Key Enabled and Value true

image-20240306-225548.png

NavPage Attributes

Following NavPageAttribute needs to be added to configure user Importer. Where “NIUserCsvIntegrator“ will be replaced by the name of any other user integrator.

  • IsTestMode: It defines whether the user Importer is in a test mode or not.
    Enable Integrator on non-live environments

    AddNavPageAttribute /Root/NI System.Customer.Integrator.IsTestMode.NIUserCsvIntegrator True

    App.config

    <add key="Core.SiteType" value="Test" />

  • CsvDelimiter: This attribute is optional and if not set the default delimiter (,) is used. (i.e. '|' for CCL)

  • ImportFilePath: This path links to the file path that the customer will drop the new user files to be uploaded and processed.

  • ImportFileBackupPath: Location where CSVs will be moved after processing is completed.

  • SchedulePeriod: Schedule period in seconds.

  • PollingInterval: This defines an interval an Integrator will poll to check if it should do work (in seconds). If unset, it will default to 5 minutes.

  • CsvTranslations: To Get translations from csv column to user property (used for collecting data).

Followings are the available user scaler properties:

  • Deactivate (X): If the column DeactivateX (aka ‘Deactivate (X)’) exists, leaving the value empty (‘’) would reactivate a user. Use ANY value for DeactivateX to deactivate a user.

  • OrgLoginId: is optional, Don't update if empty. This is unique within an Organization.

  • Password: if the user already exists and left empty - the password is not updated

  • FirstName: will not be updated if empty

  • LastName: will not be updated if empty
    EmailAddress: will not be updated if empty

  • ContactEmail: will not be updated if empty

  • ForcePasswordChange: 'True' or 'False'. Anything not 'True' (e.g. empty) for new users will be interpreted as 'False'. For updates, will not be updated if not 'True' or 'False'.

  • CanViewReports: 'True' or 'False'. Anything not 'True' (e.g. empty) for new users will be interpreted as 'False'. For updates, will not be updated if not 'True' or 'False'.

*Along with these scaler properties all active OPF fields can be used in translation.

Sample CsvTranslations:
"OrgLoginId=UniqueUserID,FirstName=FirstName,LastName=LastName,Password=DateOfBirth,ContactEmail=PersonalEmailAddress,Deactivate (X)=Deactivate (X)"

Here Left side of '=' sign is the MLS user property and right side is the csv column name.

[MLS user property]=[csv column name]

  • UserImportMode: This configuration is mandatory. Possible values are ‘Partial’ or ‘Full’. Partial mode may allow to create user with invalid regular fields.

Example NavPage Attribute Configuration

--AddNavPageAttribute /Root/NI System.Customer.Integrator.IsTestMode.NIUserCsvIntegrator True AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.CsvDelimiter "," AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.ImportFilePath "E:UserImporter\incomingFile\incomingFile" AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.ImportFileBackupPath "E:UserImporter\incomingFile\backupFile" AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.SchedulePeriod "1" AddNavPageAttribute /Root/NI System.Customer.Integrator.PollingInterval.NIUserCsvIntegrator "1" AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.CsvTranslations "OrgLoginId=UniqueUserID,FirstName=FirstName,LastName=LastName,Password=DateOfBirth,ContactEmail=PersonalEmailAddress" AddNavPageAttribute /Root/NI System.Customer.Integrator.NIUserCsvIntegrator.UserImportMode "Partial"

User Integrator Config

User importer is configured using a JSON file (i.e UserIntegratorConfigFile.json). It has multiple sections which are called activities. Every configured activity is being executed in order (can’t be changed via configuration). The order the activities are executed is the order this document describes the activity configurations.. Every section is described below, also a sample configuration file is attached.

The location of the configuration JSON file will be inside organization file collection. (i.e E:\LiveCanada\File_Collection\NI\NI\1\Configuration\Integration\User\UserIntegratorConfigFile.json)

Data Consistency Check Configuration (Optional)

Sample Configuration:

Currently this configuration section is empty.

Value Mapping Configuration (Optional)

Sample Configuration:

If a user field value needs to be mapped to a different value then this section can be configured. Value mapping is only available for single-choice-string OPFs.

FieldName: Name of the field to be mapped (case-sensitive).

MappingMode: It can be three different types.

  • Complete (default): All the OPF values must have a mapping in the value mapping file. Otherwise it will be considered as configuration error.

  • WarnIfMappingIsMissing: Ignore any missing OPF value mapping and try to set the OPF value as is. Add a warning to the notification.

If many OPF choices are missing in the mapping, it can result in a lot of warning messages (one per missing choice). This mapping mode is meant for small sets of OPF choices.

  • IgnoreMissingMappings: Ignore any missing mapping and try to set the OPF value as is. Don’t add a warning/error to the notification.

MappingFileName: This property points to the location of the file that contains the mapping of SourceValue and TargetValue. SourceValue and TargetValue are case-sensitive.

Sample ValueMappingRank.json:

Data Sanitation Configuration (Optional)

Sample Configuration:

The user importer can configure OPFs/User properties & SanitizeValues (array) which need to be ‘sanitized’.

FieldName: The name of the field to be sanitized. (Case Insensitive)

ReduceToSingleWhiteSpace: If this value is true, multiple white spaces will be trimmed to single whitespace. (i.e “Mobile Phone“ => “Mobile Phone“). This field is optional. Possible values are ("true" or “false“). If doesn’t exist, will be interpreted as 'false'.

SanitizeValues: It contains the original and replacement values for the field.

 

If an empty SanitizeValues array is provided for any field then only default sanitization is applied.

Default Sanitations (Original/Replacement values are case-sensitive):

  • Original = "(", Replacement = "["

  • Original = ")", Replacement = "]"

  • Original = ",", Replacement = "-"

  • Original = "\", Replacement = "/"

  • Original = ":", Replacement = "-"

  • Original = "|", Replacement = "/"

Reduce To One Email Address Configuration (Mandatory)

Sample Configuration:

If there are multiple email address provided for one user AND those are separated by the MultipleEmailsDelimiter only the first email is imported if this configuration is present.

Data Formatting Configuration (Optional)

Sample Configuration:

Data formatting takes an input value and formats it according to the configuration. For every field which needs formatting the expected input format and output format is needed. The expected input format may be optional and that depends on the data type. Above configuration shows formatting of a date field.

Supported data types:

  • String

  • DateTime

  • Number

Data Assembler Configuration (Optional)

Sample Configuration:

 

The data assembler activity is able to modify user property fields dynamically or create values using other user property fields. Every user property can be used to aggregate data for another or the same field.

For example, Above configuration adds a prefix to an user property "OrgLoginId" which consists of 3 different segments, “Operation Line” “-“ & “OrgLoginId“. If IsField=true, that means it will get the value provided in the csv, Otherwise whatever the value provided in the configuration value property it will consider that. Please consider that any previous data manipulation will have taken effect (i.e. DataSanitization). It may not be the original value from the CSV anymore.

Reset Fields To Default If Empty Configuration (Mandatory)

Sample Configuration:

If no value (null/empty) is provided for a user field listed in the ResetFieldsToDefaultIfEmpty array, it resets the user property (from no value) to the field’s default value (dependent on the data type - see below).

Any field not included in ResetFieldsToDefaultIfEmpty and no value was provided (null/empty), will not be updated (ignored).

 

If we want to set null/empty value to any field, those fields must be included in ResetFieldsToDefaultIfEmptyConfiguration. That means, if any organization uses deactivation field (Deactivate (X)) that needs to included in this configuration.

Default values for different user property types:

  • User scalar property (string): empty string (except any Primary Identifying Field)

  • OPF (single-choice-string): first choice

  • OPF (string): empty string

  • OPF (int): not supported yet (we don’t have a default value for int yet)

Data Validation Configuration (Mandatory)

Sample Configuration:

This activity validates all parsed user objects according to configuration and LMS specification (i.e. single choice field must have the proper value).

There are different types of user fields (These are expected to stay even if empty)

  • IdentifierFields (Mandatory): used to uniquely identify a user (i.e. OrgLoginId, login email address). Currently it supports one identifier field.

  • Critical field which needs data validation and would stop processing the whole user if it contained invalid data (skip user). Continue with any other users though.

  • Regular fields: ‘Partial' user import mode may allow to create user with invalid regular fields. In case of 'Full’ import mode it would stop any user to be imported if there is any invalid regular field(s).

  • Password fields used for password creation only.

Every field must have a Name and Type. Datatypes supported by this version are: String & EmailAddress If any other type is provided that is converted to string automatically.

 

List of all exact names for user scalar properties which can be used:

  • IdentifierFields": "OrgLoginId"

  • CriticalFields": "Rank", "Vessel", "FirstName", "LastName"

  • RegularFields: "EmailAddress", "ContactEmail", “Deactivate (X)”, “UserLanguage”, “CanViewReports”, “ForcePasswordChange”, “Password“

  • PasswordFields: "Password"

Password Configuration (Mandatory)

Sample Configuration:

The user password needs to be created/initialized for new/reactivated users.

UserReactivationAction When an user gets reactivated then password of that user depends on this configuration. Values can be ForcePasswordChange, Random, None.

If UseRandomPassword is set to true then it will create random password and ignore the PasswordFormat. Otherwise it creates password with the combination of other fields or values.

All the values inside the PasswordFormat will be concatenated for password creating for new user only. If IsField=true, that means it will get the value provided in the csv (Also can be modified in previous activities), Otherwise whatever the value provided in the configuration value property it will consider that.

Auto User Deactivation Configuration (Optional)

Sample Configuration:

UserFilterFieldNames array takes the field name to filter the users on which auto deactivation will be applied if filtered users are not part of the csv import, where field must be a Single Choice String (ScS).

Also, Deactivation users count must be less than less than MaxUsersToDeactivate

Threshold Configuration (Mandatory)

Sample Configuration:

Thresholds This configuration is designed to monitor various user import related operations and to take different actions based on the configuration if certain thresholds are exceeded (greater).

Action can be 3 different types (GenerateWarning, StopImport or None):

  • GenerateWarning: If the configured threshold exceeds it only generates warning using mail but doesn’t stop import.

  • StopImport:: If the configured threshold exceeds it stops import immediately.

  • None: If the configured threshold exceeds it only logs (log file) the issue but doesn’t stop import.

Possible Threshold Names are:

  • MaxUsersPerImport

  • MaxDeactivateUsers

  • MaxNewUsers

  • MaxOrgProfileValueUpdates

  • MaxReactivateUsers

  • MaxInvalidUsers

User Importer Configuration (Mandatory)

Sample Configuration:

 

BatchSize (Mandatory): It defines how many users will be processed within one transaction - used for performance tuning only.