Configuration

The new user importer tool is completely configured via Org NavPage /wiki/spaces/CMD/pages/5432397461. (Page attributes, NOT Link attributes).  It is available as of 3.4.0.

App.Config

Core.UserImporterInterval - (in min) set the interval to run the automatic user importer thread. Default: 15 min (3.9.0+: 120min). Only used if AUTO-USER-IMPORT is enabled.

NavPage Attributes

There are new attributes used to configure the User Importer for an organization:

"System.Customer.UserImportFilePath"  -- Mandatory attribute to enable AUTO-USER-IMPORT thread. Used to specify the path that will be used by the User Importer Background Thread when searching for a file.  NOTE: This must be an empty folder with no sub folders (one configured unique folder per org) as the thread tries to process ALL files in this folder and sub folders. The value must be a fully qualified path (e.g. D:\MarineLS\CMG\UserUploadFiles). Make sure to create the same folder on any hotbackup server or we get error messages like 'Can't get files from directory'.

"System.Customer.UserImportBackupFilePath" – Mandatory attribute to enable AUTO-USER-IMPORT thread. This is the folder that files are backed up in.  Files in the ImportFilePath folder are deleted after they are process.  The Back up folder is cleaned out files of a certain age (configured in the app.settings) by the UserImporter thread.  NOTE: This must be an empty folder with no sub folders as this folder is cleaned out (deletion) by the thread at regular intervals.  The value must be a fully qualified path (e.g. D:\MarineLS\CMG\PreviousUserUploadFiles). Make sure to create the same folder on any hotbackup server or we get error messages like 'Can't get files from directory'.

"System.Customer.UserImportEmailNotificationContacts" - Mandatory attribute to enable AUTO-USER-IMPORT thread. This is the email address(es) that the User Import reports successes and failures to. Separate multiple email addresses with comma (',').

"System.Customer.UserImportBatchSize" – Mandatory attribute.  This is represented as an integer, and represents the number of users/rows to process in a batch run.  This is included for performance reasons.

"System.Customer.UserImportUsePasswordOnCreate" - Optional attribute (applicable for New Users only). A boolean to configure if the password column in the CSV should be used (New Users only).  True (default) will use the password field as initial password. False, will ignore the password in the CSV file. See System.Customer.UserImportUseRandomPasswordIfNotProvided for detailed behaviour in combination with other configuration options.

"System.Customer.UserImportUseRandomPasswordIfNotProvided" - Optional attribute (applicable for New Users only). A boolean flag to configure whether a random password (Guid) should be generated (New User only) if no password (empty/white spaces only) is provided in the password column of the CSV file. If a random Guid is set as password, no password complexity rules are enforced/checked for the new user. Default value (i.e. missing attribute): false. 
Note: The attribute is ignored if System.Customer.UserImportClassName = CUKUserImporter. That is, don't create any user with a random password - always create password from Date of Birth.

"System.Customer.UserImportNewUserPasswordFormat" - Optional attribute.  A string that can be used to configure an Org's initial password.  Accepts all AlphaNumeric characters, special symbols and Profile index keys (e.g. LastName).  Can NOT use char '+' as that is used to parse the field.  Additionally, you can specify casing by making the key to be used all caps or lowercase.  If the Key not in all caps or lowercase, it will capitalize the first letter. Example Values: "LastName+123!" (yields "Sample123!") or "123+firstname+!!!+LASTNAME+321" (yields "123sample!!!SAMPLE321").  NOTE: As of 3.6.3 The base UserImporter now supports "DateOfBirth" in this field (value).  You must pass a valid DateTime format as a "predicate".  (E.g.  'DateOfBirth(dd-MMM-yyyy)' , 'DateOfBirth(dd-MMM-yyyy,dd-MM-yy)').  Additionally DateOfBirth does not allow the casing changes like other field (e.g. LastName). See  PSDEV-439 - Getting issue details... STATUS

DateOfBirth(pwd output date format,optional:input date format provided in CSV file): The first parameter specifies the format used to generate the initial user's password (output). The (optional) 2nd parameter can be used to specify the date format of the birth date in the CSV file. This is required if the date format in the CSV file is not in ISO standard. The input date format helps the user importer to parse the date correctly. For example, if the date format in the CSV file is 'dd-MM-yy', the automatic parsing mechanism would automatically assume a date format 'MM-dd-yy' and therefore would switch day and month because the server's operating system is set to format dd-MM-yy.
See System.Customer.UserImportUseRandomPasswordIfNotProvided for detailed behaviour in combination with other configuration options.

"System.Customer.UserImportExpireInitialPassword" -  Optional attribute.   A boolean to configure if the initial password needs to be immediately force changed.  True will force change the password on first login. To make ForcePasswordChange (or expire initial password) work, the following 2 NavPage attributes System.Security.PasswordExpiry.ExpiryDuration and System.Security.PasswordExpiry.ExpiryWarningDuration must be configured (see Attribute Definitions for details).

.

Another way of interpreting the matrix is the following password creation flow logic:

"System.Customer.UserImportClassName" -  Optional attribute.  This is the name of the class that inherits from the default UserImporter class.  (e..g "CUKUserImporter" (uses DateOfBirth) or "CarnivalUserImporter"  => CUKUserImporter.cs : CarnivalUserImporter.cs : UserImporter.cs) NOTE: As of 3.6.3 The CUKUserImporter no longer exists.  The CarnivalUserImporter has consolidated all the code and now supports DateOfBirth in passwords.

"System.Customer.UserImportCsvDelimiter" – Optional attribute.  Used to override the default CSV delimiter (',') with another char. (e.g. '|' for CCL. String used as Regex).  Because attributes are stored as a string, only put the character itself in the Value, not the single quotes.

"System.Customer.PreserveOrgLoginIdOnDelete" - Optional attribute.  IMPORTANT: Because the check on deleted users requires verifying the OrgLoginId or Email (which are nulled out by default on delete) you need to set this attribute to 'true' in order to bring back deleted users.  Otherwise a new user will be created.

"System.Customer.UserImportProfileTranslations" – Optional attribute.  Stores the mapping for user Profile Indexes to be used by the user importer. If this is not set, assume default names for DB keys (see below for defaults key names).  The key/value pairs need to adhere to strict naming (Attribute keys are string constants, if they don't match, importer will fail).  For example: First and Last Name cannot be in the same column. All keys must follow db table column names.  The order does not matter, as the indexing is built dynamically.  

Example Value: Deactivate (X)=Deactivate (X), OrgLoginID=UniqueUserId,FirstName=First Name,LastName=Last Name,Password=Password,EmailAddress=EmailAddress,Department=Department,Vessel=ShipLocation,Posting Rank=Posting Rank,Rank=Rank

"System.Customer.UserImportThreshold.MaxCsvRowsPerFile" – Optional attribute. Specifies the max. allowed user rows per CSV file (excl. header) for import. If the attribute is not specified, allow an unlimited number of user rows per file. If max. rows is exceeded (incl. empty lines), the file is not being processed. Auto-user import only: Move the CSV file into System.Customer.UserImportBackupFilePath, and an email report is being sent to inform the receivers that the file has not been processed. 

"System.Customer.UserImporterRankOrgProfileFieldName" - Mandatory attribute for RankBasedValuesByRank.json Stores the name of the "Rank" OrgProfile field for the org (E.g. "Rank" for Carnival, "Job Description" for BCF).

"System.Customer.UserImporterRankBasedValuesByRank" - Mandatory attribute for RankBasedValuesByRank.json This is a JSON value which maps ranks to sets of profile field values for those ranks (previously was Customer.CUK.RankBasedValuesByRank)

• see 2016.07.07 - RankBasedValuesByRank.json for an example
• Upload the specific JSON file via "File Upload" tool on /pages/dev for mapping purpose, and the location of the upload file will be in directory \CoreWebTemporaryUpload\
• To update (or create) these values in a live system you will need access to the live server as well as a super admin for that page. Once you have that, from the Client tool on /pages/dev usage is:
                           CreateOrUpdateNavPageAttributeFromFile "/Root/Carnival/CCL/-File/templates/HomePage.cshtml" System.Customer.UserImporterRankBasedValuesByRank CarnivalRanks.json
  • The JSON file will be exported or imported from D:\MarineLMS\Carnival\CoreWebTemporaryUpload\

Customer Specific Attributes

CMG

"System.Customer.CMG.UserImportSourceToOperatingLineMapping" - Mandatory attribute for CMGUserImporter.  Maps the import source in CMG files to an Operating line.  Current configuration as of 3.7.0 is “Import Source=Operating Line:CST_MISTRAL=Costa,AID_MISTRAL=AIDA,CST_NAMES1=Costa,AID_LOGA=AIDA”

"System.Customer.CMG.DeltaUserMaxDeleteUserLimit" - Mandatory attribute for CMGUserImporter. Sets the maximum number of users allowed to be deleted by the delta logic.  This threshold should allow for a reasonable number of deletes, but prevent bad or incomplete files from being run against the delta logic.

Note: Current develop needs to have every field explicitly matched if this attribute is used.  E.g. if an OrgProfileField is not included in the translations, it will not be imported.  See  PSDEV-371 - Getting issue details... STATUS

// Generic User Import File Field Names (keys for dictionary)
private const string DeactivateUserKey = "Deactivate (X)";
private const string OrgPathKey = "OrgPath";
private const string OrgLoginKey = "OrgLoginID";
private const string LoginKey = "LoginID";   //This can be UserName or EmailAddress
private const string PasswordKey = "Password";
private const string FirstNameKey = "FirstName";
private const string LastNameKey = "LastName";
private const string EmailAddressKey = "EmailAddress";
private const string ContactEmailKey = "ContactEmail";
private const string CanViewReportsKey = "CanViewReports";
private const string ForcePasswordChangeKey = "ForcePasswordChange";

Notes: 

• OrgProfileFields are grabbed from that organization's repository, so the CSV header name must match the OrgProfileField DB Key (e.g. Vessel, Role, etc...), otherwise that column will be ignored (as the importer won't know what to do with it)
• To make ForcePasswordChange work, the following 2 NavPage attributes System.Security.PasswordExpiry.ExpiryDuration and System.Security.PasswordExpiry.ExpiryWarningDuration must be configured (see Attribute Definitions for details).

Example of a configured org:

Example CSV:

Default CSV Format:

Note: Order does not matter.

Deactivate (X), OrgPath, OrgLoginId, LoginId, Password, FirstName, LastName, EmailAddress, ContactEmail, CanViewReports, ForcePasswordChange, [OrgProfileFieldName]*

Note:

Deactivate (X): The first column indicates whether a user should be deleted (D) or deactivated (X). Leave empty to create/update and activate the user
OrgPath: is optional if no OrgLoginId has been set. Add user to this organization
OrgLoginId: is optional if LoginId has been set. Don't update if empty. Used as a secondary look up key to find an existing user (if LoginId is empty)
LoginId: can be user name or email address. Optional if OrgLoginId has been set. Used as a primary look up key to find an existing user
Password: if the user already exists and left empty - the password is not updated
FirstName: will not be updated if empty
LastName: will not be updated if empty
EmailAddress: will not be updated if empty
ContactEmail: will not be updated if empty
CanViewReports: 'True' or 'False'. Anything not 'True' (e.g. empty) for new users will be interpreted as 'False'. For updates, will not be updated if not 'True' or 'False'.
ForcePasswordChange: 'True' or 'False'. Anything not 'True' (e.g. empty) for new users will be interpreted as 'False'. For updates, will not be updated if not 'True' or 'False'.
OrgProfileFieldName: all active OrgProfileFieldNames

Notes:

• The importer NEEDS that the headers are in the file.  The user importer uses the first line of the CSV to create it's mapping to the DB.  If the keys in the CSV do not match the keys in the previous section, then a mapping must be added to the translation attribute (also above).
• Default Passwords - currently uses the user guid. force change is true for newly created user unless a password is specified in the file or the attribute.  Passwords will never update for existing users.

Creating a New User VS Updating an Existing User

The order of priority for finding an existing user is as follows. If a different implementation is required, it must use a custom integration (see next section):

1) UserName - Unique, generated by code.   Never able to update or change.

2) OrgLoginID - If an OrgLoginID is found, but the email is different; the email could be updated. 

3) Email - If an email is found, but doesn't match the OrgLoginID, an error is generated.

Default Existing User check:

Auto User Importer (via background thread)

To enable auto user import the following 3 NavPage attributes must be configured:

• System.Customer.UserImportFilePath
• System.Customer.UserImportBackupFilePath
• System.Customer.UserImportEmailNotificationContacts

App.config Core.UserImporterInterval specifies how often the user importer checks for new files in System.Customer.UserImportFilePath. Default value is 15min (3.9.0+: 120min).

Once an org has been configured, all that is left if for the customer/client to drop their password encrypted zip file or straight up CSV file into the directory set in the attribute System.Customer.UserImportFilePath (see above for details).

Client Steps:

1. Place zip file or CSV file in the directory that was configured by the attribute: System.Customer.UserImportFilePath.
2. Done!

The background thread will check if there is a file in that directory.  Any file found will be processed as follows:

1. If it is a zip file, it is extracted.
2. The CSV file is processed (users are imported)
3. The zip/CSV file is moved to the configured directory (set by the attribute: System.Customer.UserImportBackupFilePath) for historical purposes.
4. An email is sent reporting success/warnings/errors to email addresses configured by the attribute: System.Customer.UserImportEmailNotification.
5. The backed up zip/csv file from (3) is deleted after 7 days, unless configured otherwise in the App.Config.

Manual User Importer

The new user importer can be manually used via Core client command importUsers or pages/dev Upload User tab. If no order is specified (as per above) default order is assumed.

Usage: importUsers [org path] [file path] [update: true/false] [reactivate: true/false]

Example usage: 

importUsers /Root/Moran "c:\test\newusers.csv" true true 

Note: If ImportUsers does not work, try ImportUser. 

New User Importer

The current user importer is less flexible than we would like it to be and some functions like sanitization requires external Powershell preprocessing of a user csv file. In order to have a better set of features we have decided to implement the user importer from scratch.

The current specification of the new user importer can be found at https://docs.google.com/document/d/1hfPRyPDzbaCiTgTM6BCOzVPCOn31l5oFXEF_hjmQ3qs/edit