Objective
This document serves as a reference guide for configuring the Compliance.ai Service Connector (CAI SC).
Compliance.ai Service Connector (CAI SC) Configuration
CAI SC is designed to facilitate the exchange of information between CAI and external systems, and to automate various processes and workflows.
Understanding the SC configuration structure enables administrators to define how the integration should function and specify the necessary details to establish connections, authenticate access, and determine the transformation and publication actions to be performed.
The CAI SC Configuration file contains various settings and parameters used to configure and customize the behavior of a single CAI SC instance.
The configuration file includes settings related to API access & authentication, content transfer protocols, publishing destinations, task definitions, content mapping and transformation settings. These settings can be modified according to the specific requirements and use cases.
CAI SC Operational Mode and Job descriptions
Each CAI SC instance can perform a specific job, based on the CAI SC mode of
operation:
-
Job type 1: Transform & Publish newly published regulatory documents, annotations, obligations and tasks from CAI to supported 3rd party GRC/SFTP/DMS systems
-
Job type 2: Transform & Publish newly user defined labeled documents/annotations to 3rd party GRC/SFTP/DMS
-
Job type 3: Update and apply user defined CAI labels to newly published versions of regulatory documents (from previous versions)
-
Job type 4: Bulk Associate (import) user provided mapping content to CAI Document and/or Annotation labels
-
Job type 5: Externally & dynamically update document/obligation level workflow tasks and associated attributes to manage various workflow attributes
SC Configuration Parameters and Settings
Note: Values such as "TODO_TENANT_NAME" reflect parameters that need to be updated with actual values during the configuration of each CAI SC instance, specific to each deployment environment.
Parameter |
Description |
Sample value |
Type |
---|---|---|---|
config_location |
Name of the CAI SC instance configuration file for the environment (on AWS S3). |
Managed by CAI |
String |
operational_mode |
CAI SC instance operational mode: Each SC instance has a unique operational mode. Possible values for Operational Mode:
|
UPDATE_LABEL_VERSION |
String |
api_key |
CAI provided API key for accessing CAI SC functionality
|
Managed by CAI |
String |
doc_limit_count |
Maximum number of documents to process as part of each CAI API run. |
200 |
String |
how_long_to_postpone_enforcements |
Number of days to postpone enforcement action processing while CAI retrieves enhanced meta data (to prevent double publication) . |
7 |
String |
max_records |
Maximum number of tasks being processed per SC run |
200 |
String |
LogLevel |
Log level for debugging purposes or providing information. |
DEBUG or INFO |
String |
frequency |
Frequency of SC execution (Every 15 minutes to 1 once a day) |
multi |
String |
publish_mode |
Enable and use only when SC is setup to retrieve tasks and docs |
docs_and_tasks |
String |
max_execution_time |
Maximum execution time in minutes. |
15 |
String |
refresh_jurisdictions_cache_days |
Frequency (days) to refresh jurisdictions cache |
400 |
String |
enforcement_category_doc_types |
Document types associated with enforcement actions and penalties. |
["Enforcement Document", "Enforcement Action", "Cease and Desist", "Civil Money Penalty", "Complaint", "Consent Order", "Decision", "Enforcement", "Final Decision", "Final Notice", "Final Order", "Financial Sanction", "Monetary Penalty", "Notice of Intent", "Prohibition from Banking", "Prosecutions", "AWCs (Letters of Acceptance, Waiver, and Consent)"] |
List |
resource_category_doc_types |
Document types associated with resource categories. |
["CFR", "CFR Navigation", "US Code", "US Code Navigation", "US Public Law", "US Public Law Navigation", "State Code", "State Code Navigation", "Statute", "Statute Navigation", "Guide Navigation", "Guide", "Rule Navigation"] |
List |
CAI_doctypes_cached |
File name for the cached CAI document types for
performance purposes |
CAI_doctypes_cached.csv |
String |
CAI_jurisdictions_cached |
File name for the cached CAI jurisdictions for performance purposes (refreshed periodically every 24 hours by CAI SC) |
Managed by CAI |
String |
PUBLISH_NEW_LABELS |
See section PUBLISH_NEW_LABELS |
{.. } |
Dictionary |
UPDATE_LABEL_VERSION |
See section UPDATE_LABEL_VERSION |
{.. } |
Dictionary |
DOC_LABEL |
See section DOC_LABEL |
{.. } |
Dictionary |
AB |
Connection and mapping Instructions for AuditBoard See section AB |
{.. } |
Dictionary |
SFTP_CSV |
Connection and mapping Instructions for any system using SFTP published content that is CSV formatted. See section SFTP_CSV |
{.. } |
Dictionary |
SFTP |
Connection and mapping Instructions for any system using SFTP published content that is JSON formatted. See section SFTP |
{.. } |
Dictionary |
cached_regulations_file |
File name for the cached CAI regulations for performance
purposes |
Managed by CAI |
String |
CAI_Org_Users_cached |
File for caching user email & team subscribers/licenses for performance purposes (Refreshed periodically every 24 hours by CAI SC) |
Managed by CAI |
String |
gdrive_access_info |
Access information for publishing content to a Google Drive folder |
Managed by CAI |
List |
upload_rules |
When publishing content, what type of content to upload and where (SFTP, S3, GDRIVE, etc.) |
"upload_rules": {"output_doc_name": ["SFTP","S3"], "outputFile_obl": ["SFTP","S3"], "outputFile_ann": ["SFTP","S3"], "outputFile_task": ["SFTP","S3"], "outputFile_recent":["SFTP","S3"], "outputFile_obl_recent":["SFTP","S3"], "outputFile_ann_recent": ["SFTP","S3"],"outputFile_task_recent": ["SFTP","S3"], "outputFile_auth_recent":["SFTP","S3"], "outputFile_auth_l":["SFTP","S3"] } |
Dictionary |
Conditional rule based attribute mapping: CAI_matching_conditions
The "CAI_matching_conditions" section provides a set of conditions that will be applied to any CAI SC attributes being mapped. Specific CAI attributes can be mapped based on conditional rules.
Each set of conditional rules enables rules based attribute selection and supports a flexible mapping process:
If the CAI attributes don’t meet conditions outlined in Rule 1 (r1) ->
apply Rule 2 (r2). And if CAI attributes don’t meet the conditions outlined
in Rule -> apply r3, so forth and so on.
Each mapped content type (Document, Obligation, Annotation, etc.) can have its own "matching conditions" collection of rules.
CAI SC also supports publishing multiple records for each
annotation/obligation entry inCAI that
has one or more associated labels.
This enables users to make an annotation inCAI,
apply more than one label to an annotation/obligation -> publish the
results as 'individual/unique' records (one entry per label).
While the same annotation and/or obligation is being assessed by multiple
teams/departments within an organization inCAI,
they may want to take distinctly different actions on each
obligation/annotation on a per team/department basis when published to an
external/downstream (for example GRC) system. So they may need the same
obligation/annotation to be published as multiple/unique entries.
To instruct an SC instance to create & publish unique records per each
label associated with an annotation/obligation assessment, apply the
following updates to CAI_matching_conditions section of the SC
configuration, under the Document_Mapping, Obligation_Mapping and/or
Annotation_Mapping sections of the SC configuration:
"CAI_matching_conditions": {
"unique_row_per_label":"true",
"unique_identifier_attribute":"CAI Review ID",
"unique_identifier_prefix": "BU:",
"Include_in_output":"false"
}
-
CAI_matching_conditions.unique_row_per_label accepted values : True, true, False, false.
-
Definition: Determines if SC should generate a unique output row per matching unique label (multi-row vs. single row). Default value (If not specified): "false"
-
CAI_matching_conditions.unique_identifier_attribute accepted values: Name of any attribute mentioned in the Obligation_Mapping or Annotation_Mapping sections of this service connector config document that uniquely identifies each published row for mapping purposes. Default value (If not specified): "guid"
-
Definition: The label id for the associated label is appended to the beginning of the unique_identifier_attribute to generate a unique id for the published row. The combination of attribute label_id + "_" + unique_identifier_attribute attribute is used to uniquely identify each published row. SC uses this new value to maintain a unique map between eachCAI published record and the corresponding destination system data.
For example, if the unique_identifier_attribute value for a specific annotation is evaluated as "123_456" and the associated unique label id is "789", the mapped/published unique identifier value will be "789_123_456"
-
CAI_matching_conditions.unique_identifier_prefix accepted values: The "label name" prefix that triggers the unique publication consideration.
-
Definition: Any label associated with an obligation or annotation that matches the prefix requirement will be published as a unique row. For example, if a label name is "BU: xyz", that label will be considered for multi-row publication.
If more than one associated label matches the prefix specified, they will each be treated as a unique row. For example, if an annotation is associated with labels 3 labels: "BU: label1", "BU: label2" and "label3", 2 output rows will be published for the the same annotation data (uniquely identified using the results of unique_identifier_attribute assessment)
-
CAI_matching_conditions.include_in_output: Set to "false". Default value (If not specified): "false"
-
Definition: Setting the value to "false" instructs SC to use this attribute for instruction purposes, and not for attribute mapping. Informs SC that this entry (CAI_matching_conditions) is not a mapped attribute.
Sample Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
section |
Conditions for the "section" parameter |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "not equal to","value": ""},{"group": "r3","condition": "equals","value": "ANY"}] |
Array |
framework_item_custom_select5_option_id |
Conditions for the "framework_item_custom_select5_option_id" parameter |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "equals","value": "ANY"}] |
Array |
subsection |
Conditions for the "subsection" parameter and for different rules |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "not equal to","value": ""},{"group": "r3","condition": "equals","value": "ANY"}] |
Array |
uid |
Conditions for the "uid" parameter and for different rules |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "not equal to","value": ""},{"group": "r3","condition": "not equal to","value": ""},{"group": "r4","condition": "equals","value": "ANY"}] |
Array |
name |
Conditions for the "name" parameter and for different rules |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "not equal to","value": ""},{"group": "r3","condition": "equals","value": "ANY"}] |
Array |
custom_text4 |
Conditions for the "custom_text4" parameter and for different rules |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "equals","value": "ANY"}] |
Array |
framework_item_custom_select4_option_id |
Conditions for the "framework_item_custom_select4_option_id" parameter and for different rules |
[{"group": "r1","condition": "not equal to","value": ""},{"group": "r2","condition": "equals","value": "ANY"}] |
Array |
unique_identifier_prefix |
"label name" prefix that triggers the unique publication consideration. |
"unique_identifier_prefix": "BU:" |
Key-Value paid |
unique_identifier_attribute |
Name of any attribute mentioned in the Obligation_Mapping or Annotation_Mapping sections of this service connector config document that uniquely identifies each published row for mapping purposes. Default value (If not specified): "guid" |
"unique_identifier_attribute":"CAI Review ID" |
Key-Value paid |
unique_row_per_label |
Publishing multiple records for each annotation/obligation entry in Compliance.ai that has one or more associated labels. |
"unique_row_per_label":"true", |
Key-Value paid |
include_in_output |
Ignore this attribute in the mapped output |
"include_in_output":"false" |
Key-Value paid |
Dynamically updating attributes mapped values: Attribute_Translate
Attribute translation mapping: Provides the ability to replace the CAI provided final value for an attribute with a mapped value, enabling for consistent representation of data between CAI and external systems.
The mappings defined in "Attribute_Translate" can be used to convert values from one format to another.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
subsection |
Maps the "United States of America" to "US Federal" |
{"United States of America": "US Federal"} |
Object |
framework_item_custom_select5_option_id |
Maps options for the "REGULATION APPLICABLE?" field |
{"friendly_name": "REGULATION APPLICABLE?", "Yes": "1", "No": "2", "Y": "1", "N": "2"} |
Object |
framework_item_custom_select3_option_id |
Maps options for the "Requirement_Level" field |
{"friendly_name": "Requirement_Level", "Federal": "1", "State": "2", "Provincial": "3"} |
Object |
framework_item_custom_select2_option_id |
Maps options for the "Country" field |
{"friendly_name": "Country", "Australia": "1", "Canada": "2", "UK": "3", "US": "4"} |
Object |
framework_item_custom_select4_option_id |
Maps options for the "Cons/FinCrime" field |
{"friendly_name": "Cons/FinCrime", "Consumer": "1", "Financial Crime": "2"} |
Object |
Framework Category |
Maps options for the "Framework Category" field |
{"friendly_name": "Framework Category", "Compliance": "1", "ESG": "2"} |
Object |
Self-Assessment Result |
Maps options for the "Self-Assessment Result" field |
{"friendly_name": "Self-Assessment Result", "Pass": "1", "Fail": "2", "Other": "3"} |
Object |
Map to external attributes defined as labels: Attribute_Label_Prefix_Mapping
CAI labels can be used to map documents, annotations or obligations within documents to "custom" or external attributes. Each CAI label can have a prefix associated with its name.
CAI SC can perform the following actions:
-
Identify recently labeled artifacts (documents, annotations, obligations)
-
If "Attribute_Label_Prefix_Mapping" is configured: Strip the prefix of each label (the prefix can help determine the destination attribute name)
-
Publish the value in the label (post prefix) as the custom attribute value named in the label prefix:
-
CAI Label Prefix -> maps to the 3rd party system's custom attribute’s name
-
Additionally, the CAI Label prefix value can be mapped to another 3rd party attribute name
key -> value pairs under the Attribute_Label_Prefix_Mapping section -
CAI Label Suffix-> maps to 3rd party system custom attribute’s value
Sample Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
uid |
label suffix maps to the unique identifier attribute value in destination system |
"uid" |
String |
section |
label suffix maps to the the section attribute in destination system |
"section" |
String |
subsection |
label suffix maps to the subsection attribute in destination system |
"subsection" |
String |
name |
label suffix maps to the name attribute in destination system |
"name" |
String |
framework_item_custom_select5_option_id |
label prefix maps to regulatory applicability attribute in destination system |
"reg applicability" |
String |
framework_item_custom_select2_option_id |
The label prefix for country attribute |
"country" |
String |
custom_text4 |
The label prefix for lifecycle attribute |
"lifecycle" |
String |
framework_item_custom_select4_option_id |
The label prefix for Cons/FinCrime attribute |
"Cons/FinCrime" |
String |
Monitor and publish recently labeled content: PUBLISH_NEW_LABELS
CAI SC supports 2 modes of publishing labeled content in CA:
-
User workflow driven:
-
A user labels a group of documents and completes other CAI tasks
-
The user marks CAI tasks associated with those documents as “complete” using CAI workflow task management
-
A dependant CAI SC task then publishes the labeled content to a 3rd party system
-
To enable this mode, use the CAI workflow and create a CAI SC dependant step
-
Unattended (monitored: PUBLISH_NEW_LABELS):
-
A user labels a group of documents
-
An unattended and recurring CAI SC process monitors recently labeled CAI content and publishes them to a 3rd party solution automatically based on a predefined workflow (no user action required).
-
To enable this mode, configure PUBLISH_NEW_LABELS
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
wf_sc_auto_q_labelled_docs |
Name of the workflow that initiates the PUBLISH_NEW_LABELS Activities. To disable the PUBLISH_NEW_LABELS actions, mark the task as Done. |
wf_sc_auto_q_labelled_docs |
string |
wf_sc_publish_label |
Name of the workflow to use for monitored labels. Newly labeled content is added to this workflow automatically |
wf_sc_publish_label |
string |
alert_sc_version_link |
Name of the dynamically CAI SC managed Alert that aggregates newly published labels. If new versions of labeled content are published, they are tracked in this alert -> they will also be published to the 3rd party system. |
alert_sc_version_link |
string |
last_label_review_date_file |
Last review date file for labels. Last time that newly labeled content was checked by the CAI SC. |
CAI managed |
string |
Cache_Workflow_Info |
Flag indicating whether to cache workflow information for performance purposes (refreshed periodically every 24 hours by CAI SC) |
True |
Bool |
CAI_Org_Workflow_file |
File containing cached CAI organization workflow information |
CAI managed |
string |
Monitor and publish new versions of previously labeled content: UPDATE_LABEL_VERSION
When users label CAI content, they often want CAI SC to continue monitoring incoming content for newly published versions of the content they labeled, and for CAI SC to publish updated versions to their 3rd party systems accordingly.
UPDATE_LABEL_VERSION achieves this goal:
-
An unattended CAI SC recurring process monitors recently labeled content and publishes new versions of the content to 3rd party solutions automatically, based on a predefined workflow that requires no user actions.
-
To enable this mode, configure UPDATE_LABEL_VERSION
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Apply_Label_To_New_Versions_Task_name_prefix |
Prefix for the task name to apply labels to new versions |
__SC_Label_Other_Versions |
string |
wf_sc_auto_q_labelled_docs |
Name of the workflow that initiates the PUBLISH_NEW_LABELS Activities. To disable the PUBLISH_NEW_LABELS actions, mark the task as Done. |
wf_sc_auto_q_labelled_docs |
string |
wf_sc_publish_label |
Name of the workflow to use for monitored labels. Newly labeled content is added to this workflow automatically |
wf_sc_publish_label |
string |
alert_sc_version_link |
Name of the dynamically CAI SC managed Alert that aggregates newly published labels. If new versions of labeled content are published, they are tracked in this alert -> they will also be published to the 3rd party system. |
alert_sc_version_link |
string |
last_label_review_date_file |
Last review date file for labels. Last time that newly labeled content was checked by the CAI SC. |
CAI managed |
string |
Cache_Workflow_Info |
Flag indicating whether to cache workflow information for performance purposes (refreshed periodically every 24 hours by CAI SC) |
True |
Bool |
CAI_Org_Workflow_file |
File containing cached CAI organization workflow information |
CAI managed |
string |
relating_advanced_matching_critera |
Advanced matching criteria for relating labels. Versioned documents are linked directly by CAI based on the source’s docket for example, and/or based on this criteria. |
(this_doc.title == matched_doc.title and this_doc.agencies == matched_doc.agencies and this_doc.cai_category == matched_doc.cai_category and this_doc.location_id == matched_doc.location_id) or (this_doc.docket_id == matched_doc.docket_id or this_doc.official_id == matched_doc.official_id) |
string |
Bulk Label documents using previously mapped content: DOC_LABEL, Associate_Label_To_Doc
Using external files to load document or citation -> label pre-mapped content
If you maintain an existing labeling/tagging methodology outside of Compliance.ai, Team Edition workflows allow you to easily associate a group of labels to multiple regulatory citations/documents
This feature (Associate_Label_to_Doc) is especially useful during initial onboarding, assisting with the mass import of existing pre-mapped/pre-labeled documents/citations into Compliance.ai
Process for using External mapping files (doc id/citation -> labels) |
---|
Step 1:
|
Step 2: Save/download the new spreadsheet as a Comma Separated Value file, for example: "Label_to_Doc.csv" |
Step 3: |
Step 4: Mark your task as Complete |
Step 5:
Wait for the next Service Connector run (1-12 hours)
|
Step 6: Once complete, review the report attached to the Service Connector task, which provides the results of the processing on a per-row basis (date/time, success, failure, and error code) |
Importing external/extended attributes using external files
You can extend the attributes in Compliance.ai by importing an external Comma Separated Value (CSV) file .
Mapped values can be added as part of a bulk import, and processed using a Compliance.ai workflow.
This feature enables you to extend Compliancel.ai to match your destination system's attributes automatically.
CSV File format
The format of the CSV file is follows:
Aside from the header (column names), each row in the CSV file represents additional external attributes to be used for add/update requests.
CAI.Document.id ⇒ External_File.id => 3rdParty.uid => GRC.App.Attribute
and similarly
CAI.Document.Label ⇒ GRC.App.Attribute
Follow the steps below to configure an external file to import external attributes or to override the ones being used by Compliance.ai
Process for using external attribute files |
Step 1: Open the new spreadsheet & populate rows 2 onwards with actual data, following the schema outlined above |
Step 2: Save/download the new spreadsheet as a Comma Separated Value file, for example: "abc.csv" |
Step 3: Attached "abc.csv' to a Compliance.ai task that is associated with the document & annotations referenced in "abc.csv", using "Provide External Assessments for Annotations" workflow as your template |
Step 4: Mark your task as Complete |
Step 5: Wait for the next Service Connector run (1-12 hours) |
Step 6: Once complete, review the report attached to the Service Connector task, which provides the results of the processing on a per-row basis (date/time, success, failure, and error code) |
Bulk loading labels using External Files
You can import an extended group of labels and associate them with Compliance.ai documents, annotations and obligations using an external Comma Separated value formatted file (CSV).
Mapped labels can be added in bulk and processed using Compliance.ai workflow.
This feature enables you to rapidly add labels to your annotations and then use them for mapping.
CSV File format for bulk loading Labels -> Annotations
The format of the CSV file is as follows:
Attribute to be included |
Attribute description |
Attribute Type |
Required Attribute |
---|---|---|---|
annotation_group |
Unique group of sentences that form an annotation. As exported from Compliance.ai "annotation_groups" attribute: "first_sentence_id" + "_" + last_sentence_id" |
String |
Mandatory |
DocumentID |
Unique identifier of the Document inCompliance.ai. Associated with each Annotation Group (1 DocumentID would be associated with many Annotation Groups |
String |
Mandatory |
LabelName |
Name of Label to be associated with the annotation group |
String |
Mandatory |
LabelID |
Compliance.ai Label ID associated with the Label Name |
Integer |
Optional |
Aside from the header (column names), each row in the CSV file represents additional Label information to be used for add/update requests.
CAI.Document.id ⇒ CAI.Annotation.Annotation_Group.Label => ExternalFile.Label.Name
and similarly
CAI.Document.Label ⇒ ExternalFile.Label.Name
Follow the steps below to configure an external file for additional attributes or to override the ones being used by Compliance.ai
Process for using external label files |
Step 1: Open a new spreadsheet & populate rows with header & actual data, following the schema in CSV File format above |
Step 2: Save/download the new spreadsheet as a Comma Separated Value file, for example: "abc.csv" |
Step 3: Attached "abc.csv' to a Compliance.ai task that is associated with the document & annotations referenced in "abc.csv", using the Compliance.ai "Associate User provided Labels to Annotation Groups" workflow as your template |
Step 4: Mark your task as Complete |
Step 5: Wait for the next Service Connector run (1-12 hours depending on the frequency) |
Step 6: Once complete, review the report attached to the Service Connector task, which provides the results of the processing on a per-row basis (date/time, success, failure, and error code) |
Processing Rules: Order of execution
CAI SC applies an ordered sequence when mapping CAI attributes to external attributes.
-
External file content: CAI SC first finds any linked external file that is attached to its mapping and publication task. If such a file is present -> the Service Connector will use the attribute values present in the file to override/use those values.
If the attributes are missing or empty in the external file -> Service Connector will look for rules to map the attribute using a Label prefix (optional).
-
Labels: CAI SC will find any mapped labels (can be configured with or without prefixes). If not found, the Service Connector will look for mapped attributes (optional)
-
Internal attributes: Using any combination of CAI document or annotation attributes. Attributes can be concatenated and truncated prior to being published into the external system (optional).
-
Default values
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Document_Task_name_prefix |
Prefix for document task names |
__SFTP_CSV_DOC__ |
String |
Annotation_Task_name_prefix |
Prefix for annotation task names |
__SFTP_CSV_ANN__ |
String |
Obligation_Relevant_Task_name_prefix |
Prefix for relevant obligation task names |
__SFTP_CSV_OBLR__ |
String |
Obligation_All_Task_name_prefix |
Prefix for all obligation task names |
__SFTP_CSV_OBL__ |
String |
Auth_Source_Task_name_prefix |
Prefix for authorization source task names |
__SFTP_CSV_RES__ |
String |
Associate_Label_to_Doc_Task_name_prefix |
Prefix for task names to associate labels with documents |
Associate_Label_to_Doc |
String |
Associate_Label_to_Annotation_Task_name_prefix |
Prefix for task names to associate labels with annotations |
Associate_Label_to_Annotation_Task_name_prefix |
String |
auth_source_levels |
Number of authorization source levels |
4 |
Integer |
Regulations |
Regulations information |
|
String |
ftp_server |
FTP server address |
complianceai.files.com |
String |
ftp_uid |
FTP server username |
complianceai_uid |
String |
ftp_pwd |
FTP server password |
pwd |
String |
ftp_key_file |
Path to the key file for FTP authentication |
Managed by CAI |
String |
sftp_folder |
SFTP folder name |
|
String |
exportfiles_folder_incremental |
Folder name for incremental export files |
_Internal/incremental |
String |
exportfiles_folder |
Folder name for export files |
_Internal |
String |
destination_type |
Type of destination (e.g., sftp, ftp) |
sftp |
String |
doc_name_prefix |
Prefix for document names |
ab_Files_ |
String |
recent_document_name |
File name for the recent document CSV file |
DOCS_RECENT.csv |
String |
recent_obligation_name |
File name for the recent obligation CSV file |
OBLS_RECENT.csv |
String |
recent_annotation_name |
File name for the recent annotation CSV file |
ANNS_RECENT.csv |
String |
recent_auth_name_l0 |
File name for the recent authorization (level 0) CSV file |
AUTH_RECENT_L0.csv |
String |
recent_auth_name_l1 |
File name for the recent authorization (level 1) CSV file |
AUTH_RECENT_L1.csv |
String |
recent_auth_name_l2 |
File name for the recent authorization (level 2) CSV file |
AUTH_RECENT_L2.csv |
String |
recent_auth_name_l3 |
File name for the recent authorization (level 3) CSV file |
AUTH_RECENT_L3.csv |
String |
max_recent_days_records |
Maximum number of days to maintain 'recent' information in 'recent" files. SC purges records in the 'recent' file which have been published by the SC more than max_recent_days_records ago, each time it runs. |
2 |
String |
Document_Mapping |
See section Document_Mapping |
{..} |
complex |
Annotation_Mapping |
See section Annotation_Mapping |
{..} |
complex |
Obligation_Mapping |
See section Obligation_Mpping |
{..} |
complex |
Auth_Mapping_0 |
See section Auth_Mapping_0 |
{..} |
complex |
Auth_Mapping_1 |
See section Auth_Mapping_1 |
{..} |
complex |
Auth_Mapping_2 |
See section Auth_Mapping_2 |
{..} |
complex |
Auth_Mapping_3 |
See section Auth_Mapping_3 |
{..} |
complex |
Mapping regulatory documents: Document Mapping
Document Mapping enables transforming & publishing CAI documents. The mapping section specifies how various CAI document attributes are mapped/transformed to externally defined attributes or taxonomies.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI content that follows the CAI dot notation syntax.
Parameter-Target |
Description |
Sample Mapped Source |
Type |
---|---|---|---|
title |
Extraction rule for the document title |
["title.{'length':300}”] |
array |
description |
Extraction rule for the document description |
["summary_text"] |
array |
link |
Extraction rule for the document link |
["web_url"] |
array |
guid |
Extraction rule for the document GUID or identifier |
["id"] |
array |
Doc ID |
Extraction rule for the document ID |
["id"] |
array |
category |
Extraction rule for the document category |
["cai_category_id.{'lookup':'DocTypes'}"] |
array |
pubDate |
Extraction rule for the document publication date |
["publication_date.{'date_convert':'UTC'}"] |
array |
jurisdiction |
Extraction rule for the document jurisdiction |
["jurisdiction.{'lookup':'Jurisdiction'}"] |
array |
summary_text |
Extraction rule for the document summary text |
["“summary_text”"] |
array |
pdf_url |
Extraction rule for the document PDF URL |
["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}"], ["id"] |
array |
effective_on |
Extraction rule for the document effective date |
["rule.effective_on.{'date_convert':'UTC'}"] |
array |
comments_close_on |
Extraction rule for the document comments close date |
["rule.comments_close_on.{'date_convert':'UTC'}"] |
array |
cfr |
Extraction rule for the document CFR |
["official_id"] |
array |
topic |
Extraction rule for the document topic |
["topics.{'attrib':'name', 'scope':'*', 'delim':','}"] |
array |
agency |
Extraction rule for the document agency |
["agencies.{'attrib':'short_name' , 'scope':'*' , 'delim':','}" "], [""mainstream_news.news_source.name" "] |
array |
docket_id |
Extraction rule for the document docket ID |
["docket_ids.[0]"] |
array |
DateProcessed |
Extraction rule for the document processed date |
["{'proc_date':'UTC'}"] |
array |
Auto Summary |
Extraction rule for the document auto summary |
["summaries."0].summary_sentences.{'attrib':'', 'scope':'', 'delim':' ', 'length':20000}"] |
array |
CreatedDate |
Extraction rule for the document creation date |
[“created_at.{'date_convert':'UTC'}"] |
array |
doc_labels |
Extraction rule for the document labels |
["{'doc_label':'True'}.{'attrib':'label', 'scope':'*', 'delim':',','length':20000}""] |
array |
workflow_name |
Extraction rule for the document workflow name |
[“{'WORKFLOW_NAME':'true'}"] |
array |
location |
Extraction rule for the document location |
[“document_location.{'attrib':'title', 'scope':'*', 'delim':','}"] |
array |
Mapping user defined annotations: Annotation Mapping
Annotation Mapping enables transforming & publishing of user defined CAI annotations. The mapping section specifies how various annotation related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI annotations that follows the CAI dot notation syntax.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Date Review Started |
Date when the review started |
[“{'proc_date':'UTC'}"] |
Array |
Review Description |
Description of the review |
["sentences_text"] |
Array |
Review Status |
Status of the review |
["{'pre_val':'In Progress'}"] |
Array |
Analysis |
Type of analysis |
["{'pre_val':'Annotation Review'}"] |
Array |
Policy Change Overview |
Overview of policy changes |
["labels.{'attrib':'label', 'scope':'*', 'delim':',','length':20000}"] |
Array |
Comments |
Comments related to the review |
["comment_threads.[*].comments.{'attrib':'richtext', 'scope':'', 'delim':',' , 'l':20000, 'attrib_type':'richtext'}”] |
Array |
Type of Impact |
Type of impact |
["{'pre_val':'Policy Change'}"] |
Array |
CAI Regulatory Intelligence Items (GUID) |
GUIDs of CAI regulatory intelligence items associated with the review |
["{'service_var':'document_id'}”] |
Array |
CAI Review ID |
Unique ID of the CAI review |
["first_sentence_id"], ["{'pre_val':'_'}"], ["last_sentence_id"] |
Array |
DateProcessed |
Date when the item was processed |
["{'proc_date':'UTC'}"] |
Array |
guid |
GUID of the review |
["first_sentence_id"], ["{'pre_val':'_'}"], ["last_sentence_id"] |
Array |
Mapping obligations: Obligation Mapping
Obligation Mapping enables transforming & publishing of both automated and user defined CAI obligations . The mapping section specifies how various obligation related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI obligations that follows the CAI dot notation syntax.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Date Review Started |
Start date of the obligation review |
[“{'proc_date':'UTC'}”] |
Extraction Rule |
Review Description |
Description of the obligation review |
["”sentences_text'”] |
Extraction Rule |
Review Status |
Status of the obligation review |
[{'pre_val':'In Progress'}"] |
Extraction Rule |
Analysis |
Indication that it is an obligation review |
[{'pre_val':'Obligation Review'}"] |
Extraction Rule |
Policy Change Overview |
Overview of policy changes |
["labels.{'attrib':'label', 'scope':'*', 'delim':','length':20000}'"] |
Extraction Rule |
Comments |
Comments related to the obligation review |
["comment_threads.[*].comments.{'attrib':'richtext', 'scope':'', 'delim':'n' , 'l':20000, 'attrib_type':'richtext'}'"] |
Extraction Rule |
Type of Impact |
Type of impact (e.g., policy change) |
[{'pre_val':'Policy Change'}"] |
Extraction Rule |
CAI Regulatory Intelligence Items (GUID) |
GUID of CAI Regulatory Intelligence items |
[{'service_var':'document_id'}"] |
Extraction Rule |
CAI Review ID |
ID of the CAI review, including first and last sentence IDs |
["first_sentence_id', {'pre_val':'_'}, 'last_sentence_id'"] |
Extraction Rule |
is_system_obligation |
Flag indicating if it is a system obligation |
["is_system_obligation'"] |
Extraction Rule |
DateProcessed |
Processed date of the obligation |
[{'proc_date':'UTC'}"] |
Extraction Rule |
title |
Title of the main document |
["main_doc_attrib.title.{'length':300}'"] |
Extraction Rule |
agency |
Agency responsible for the obligation, including short name and news source name |
["main_doc_attrib.agencies.{'attrib':'short_name' , 'scope':'*' , 'delim':','}"], ["main_doc_attrib.mainstream_news.news_source.name"] |
Extraction Rule |
effective_on |
Effective date of the main document |
[“main_doc_attrib.rule.effective_on.{'date_convert':'UTC'}”] |
Extraction Rule |
guid |
GUID or identifier of the obligation, including first and last sentence IDs |
["first_sentence_id', {'pre_val':'_'}, 'last_sentence_id"] |
Extraction Rule |
Mapping to Resource type content
Mapping top-level Resource content: Auth_Mapping_0
CAI Authoritative top-level source type regulatory content are “Resource” content in CAI..
Authoritative Source mapping (or Auth mapping) enables transforming & publishing of user defined CAI resource type content. The mapping section specifies how various resource content related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
The top-level content set is related to other nth level resource content, referenced as “Auth_Mapping_n '' in the configuration template following the CAI dot notation syntax.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI resource content that follows the CAI dot notation syntax.
CAI Auth resource documents are linked together based on a parent -> child relation:
Auth_Mapping_0
-> Auth_Mapping_1
-> Auth_Mapping_2
…
-> Auth_Mapping_n
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
source name |
Name of the information source |
["title.{'length':300}"] |
Extraction Rule |
source description |
Description of the information source |
["summary_text"] |
Extraction Rule |
link |
Link to the source, including a pre-value and the ID |
["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}', 'id"] |
Extraction Rule |
source tracking Id |
Tracking ID of the source |
["id"] |
Extraction Rule |
CAI name |
Name of the CAI (Compliance AI) |
["official_id"] |
Extraction Rule |
Source Version |
Version of the source, including the publication date |
["publication_date.{'date_convert':'UTC'}"] |
Extraction Rule |
docket_id |
Docket ID associated with the source |
["docket_ids.[0]"] |
Extraction Rule |
guid |
GUID or identifier of the source |
["id"] |
Extraction Rule |
Doc ID |
ID of the document |
["id"] |
Extraction Rule |
DateProcessed |
Processed date of the source |
["{'proc_date':'UTC'}"] |
Extraction Rule |
Mapping 2nd level Resource content: Auth_Mapping_1
CAI Authoritative 2nd level resource type regulatory content are “Resource” content in CAI that are linked (children) of other resource content.
Authoritative Source mapping (or Auth mapping) enables transforming & publishing of user defined CAI resource type content. The mapping section specifies how various resource content related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
The 2nd-level content set is related to other nth level resource content, referenced as “Auth_Mapping_n '' in the configuration template following the CAI dot notation syntax.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI resource content that follows the CAI dot notation syntax.
The resource content is related to other nth level resource content, referenced as “Auth_Mapping_n ''.
For example, a specific level 1 resource content can be mapped to the level
0 (top level) resource content using the “uplevel” indicator based on the
CAI dot notation syntax.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Topic Name |
The name of the topic |
["title"] |
Array |
Topic Description |
The description of the topic |
["summary_text"] |
Array |
link |
The link to the topic |
["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}"],["id"] |
Array |
Topic Tracking Id |
The tracking ID of the topic |
["id"] |
Array |
CAI name |
The name of the CAI (Compliance.ai) |
["official_id"] |
Array |
Source Version |
The version or publication date of the source |
"publication_date.{'date_convert':'UTC'}" |
Array |
Source Description |
The description of the parent source |
["parent", "summary_text"] |
Array |
Source ID |
The source ID of the parent source |
["{'uplevel':'1', 'attrib':'id'}"] |
Array |
Source Name |
The Source Name of the parent source |
["{'uplevel':'1', 'attrib':'title'}"] |
|
guid |
The GUID (Global Unique Identifier) of the source |
["id"] |
Array |
DateProcessed |
The UTC date when the source was processed |
["{'proc_date':'UTC'}"] |
Array |
Mapping ‘nth -1 level’ Resource content: Auth_Mapping_2
Mapping parameters for transforming CAI Authoritative second-level source type regulatory content (referred to as “Resource documents” in Compliance.ai).
The mapping specifies how various resource document related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
Each mapped attribute is represented as a key-value pair:
-
Key: the external (Target) field name.
-
Value: A transformation or extraction instruction for the CAI resource content that follows the CAI dot notation syntax.
The level of resource content set is related to other nth level resource
content, referenced as “Auth_Mapping_n '' in the configuration template. For
example, a specific level 2 content can be mapped to the level 1 (up one
level) content using the “uplevel” dot notation mapping. Similarly, a
specific level 2 content can be mapped to the level 0 (up two levels)
content using the “'grand_parent'” dot notation syntax.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Section Name |
Name of the section |
["title"] |
Extraction Rule |
Section Description |
Description of the section |
["summary_text”] |
Extraction Rule |
Link |
Link to the source, including a pre-value and the ID |
["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}', 'id"] |
Extraction Rule |
Section Tracking ID |
Tracking ID of the section |
["id"] |
Extraction Rule |
CAI_ID |
ID of the CAI (Compliance AI) |
["official_id"] |
Extraction Rule |
Source Version |
Version of the source, including the publication date |
["publication_date.{'date_convert':'UTC'}"] |
Extraction Rule |
Topic ID |
ID of the parent topic |
["parent.id"] |
Extraction Rule |
Topic Name |
Name of the parent topic |
["parent.title"] |
Extraction Rule |
Source Description |
Description of the source |
["{'grand_parent':'True', 'attrib':'summary_text'}"] |
Extraction Rule |
Source Name |
Name of the source |
["{'grand_parent':'True', 'attrib':'title', 'length':300}"] |
Extraction Rule |
docket_id |
Docket ID associated with the source |
["docket_ids.[0]"] |
Extraction Rule |
guid |
GUID or identifier of the source |
["id"] |
|
Mapping “nth level” Resource content: Auth_Mapping_3
Mapping parameters for transforming CAI Authoritative third-level source type regulatory content (referred to as “Resource documents” in Compliance.ai). Specifies how various resource document related fields or attributes are mapped/transformed to externally defined attributes or taxonomies.
Each mapping is represented as a key-value pair, where the key is the external (Target) field name and the value is a transformation or extraction rule for the CAI content and the associated CAI extracted resource content.
The level of resource content set is related to other nth level resource content, referenced as “Auth_Mapping_n '' in the configuration template. For example, a specific level 3 content can be mapped to the level 2 (up one level) content using the “uplevel” dot notation mapping.
Similarly, a specific level 2 content can be mapped to the level 0 (up two levels) content using the “'uplevel'” dot notation mapping (number of up-stream levels to reverse traverse).
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Sub Section Name |
Name of the sub-section |
["title"] |
Array |
Sub Section Description |
Description of the sub-section |
["summary_text"] |
Array |
Link |
Link to the sub-section |
["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}"],["id"] |
Array |
Sub Section Tracking ID |
Tracking ID of the sub-section |
["id"] |
Array |
CAI_ID |
CAI ID associated with the sub-section |
["official_id"] |
Array |
Source Version |
Version of the source |
"publication_date.{'date_convert':'UTC'}" |
Array |
Section ID |
ID of the parent section |
"{'uplevel':'1', 'attrib':'id'}" |
Array |
Section Name |
Name of the parent section |
"{'uplevel':'1', 'attrib':'title'}" |
Array |
Topic ID |
ID of the parent topic |
"{'uplevel':'2', 'attrib':'id'}" |
Array |
Topic Name |
Name of the parent topic |
"{'uplevel':'2', 'attrib':'title', 'length':300}" |
Array |
Source ID |
The source ID of the parent source |
["{'uplevel':'3', 'attrib':'id'}"] |
|
Source Name |
The Source Name of the parent source |
["{'uplevel':'3', 'attrib':'title', 'length':300}"] |
|
guid |
Unique identifier of the sub-section |
["id"] |
Array |
DateProcessed |
Date of processing the sub-section |
["{'proc_date':'UTC'}"] |
Array |
Publishing & updating Task information: Task Mapping
Publishing Task Information
Task Mapping enables transforming & publishing "current" CAI Task information as CSV formatted content. This includes support for attributes such as “task owner”, associated “workflow name”, task comments, task assignee info, task create/update dates, etc. CAI SC supports exporting task information for tasks that the SC owned task depends on.
To instruct an CAI SC instance to publish Task information, complete the following steps.
-
Add "publish_mode" parameter to the SC configuration:
"operational_mode": "SFTP_CSV",
"publish_mode" : "docs_and_tasks"
publish_mode accepted values are: "docs_and_tasks", "docs", ""
-
Configure mapping to indicate "which" Task attributes to publish, the "Task_Mapping" section needs to be configured
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
task_id |
Specifies the task ID associated with the document task. |
["document_task.task_id"] |
Array |
created_at |
Converts and retrieves the creation date of the document task. |
["document_task.created_at.{'date_convert':'TASK_DATES'}"] |
Array |
updated_at |
Converts and retrieves the update date of the document task. |
["document_task.updated_at.{'date_convert':'TASK_DATES'}"] |
Array |
doc_id |
Specifies the document ID associated with the document task. |
["document_task.doc_id"] |
Array |
workflow_name |
Retrieves the workflow name associated with the document task. |
["document_task.workflow_name"] |
Array |
workflow_id |
Retrieves the workflow ID associated with the document task. |
["document_task.workflow_id"] |
Array |
original_assignee_email |
Retrieves the original assignee's email associated with the document task. |
["assignee.email"] |
Array |
original_assignee |
Retrieves the original assignee's ID associated with the document task. |
["assignee.id"] |
Array |
first_comment |
Retrieves the first comment related to the document task in Richtext format. |
["document_task.comment_threads.[0].comments.{'attrib':'richtext', 'scope':'*', 'delim':',' , 'l':20000, 'attrib_type':'richtext'}"] |
Array |
DateProcessed |
Converts and retrieves the processed date of the document task in UTC. |
["{'proc_date':'UTC'}"] |
Array |
Updating Task Information
Once a workflow is applied to a document, users may want to dynamically change the attributes of the workflow tasks that get created, or spawn (clone) a specific task based on external information that once applied, change the nature of the tasks that need to be performed. CAI SC supports a workflow update syntax that enables editing and creating new tasks based on existing tasks (cloning tasks).
In order to support dynamically cloning a task, CAI SC completes the following processes automatically:
-
SC follows "instructions" provided as a hyperlink in the "task.description" SC task (task assigned to the SC identity). SC applies the instructions in the referenced file to perform the following operations:
-
Create copy (or copies) of the task
-
Cloned tasks can have "unique" or "shared" task attributes, such as "watchers" (notifiers), "drop down options", "assignees", "title" and "description".
-
Update the cloned items with clone specific attributes (including assignee & watcher info)
Update Task Information instruction syntax and schema
To dynamically update or clone a task, you need to create a JSON formatted "update instruction information" document.
The instruction document should be hosted on an SFTP server with a location path that is accessible by the SC identity account. CAI supports using a CAI managed SFTP server https://complianceai.files.com for this purpose. To configure and use the CAI managed SFTP server, reach out to support@compliance.ai .
Once the instruction document is uploaded, add a link to the document's location in the Description attribute of the SC assigned task in URL format, as depicted in the screenshot below.
Each update task information "operational instruction file" should be enclosed in "SC_Ops { … }".
"SC_Ops" should include an "operations [[...]]" element that should include a list of lists.
Each list item represents a "task_type" that is to be performed by the SC to update the task information.
Currently supported task_types include the following:
Parameter-Target |
Description |
---|---|
CloneTask |
Creates copies of the current document task and creates cloned tasks.
Each cloned task can have unique attribute updates using
task types ("per_clone_operations") and common attribute
updates ("shared_for_all_clones"). Each "per_clone_operations" "task_type" is evaluated and applied for each unique cloned task (specific to each cloned task) For example, while all cloned tasks can share the same "drop_down" values for all drop down items, they can have different watchers.
Instructions["operations"]["mapping"]["data"] The "data" element supports the CAI dot notation formatted syntax for determining what CAI content values/combination of attributes should be used.
Instructions["operations"]["method"]]
Instructions["operations"]["api_params"][0]["value"]
Response content format: Regardless of the method used to retrieve the information, the default result data format is a JSON formatted list of dictionaries. Here's a sample response payload: { "list_item_name": [ { "list_1_attrib_1": "value 1", "list_1_attrib_2": "value 2, "list_1_attrib_n": "value n" }, { "list_2_attrib_1": "value 21", "list_2_attrib_2": "value 22, "list_2_attrib_n": "value 2n" }, { "list_3_attrib_1": "value 31", "list_3_attrib_2": "value 32, "list_3_attrib_n": "value 3n" } ] } |
setDropDownValue |
Set the drop-down values of an existing dropdown referenced in the current document task |
list_group |
Get a list of user ids to be used in the next processes within the instruction of operations Subsequent instructions can include references to list_groups to access the data retrieved as part of this process |
setAssignee |
Set the assignee of the current document task |
setNotify |
Set the watcher(s) (Notifiers) of the current document task |
setDescription |
Set the description of the current document task |
setTitle |
Set the title of the current document task |
update_taskdata_task_name_prefix |
Prefix of tasks in CAI workflow that instruct the SC instance to perform user defined changes to the workflow task. The instructions include various capabilities such as duplicating (cloning) tasks, changing assignee, dynamically changing drop-down values, etc. The actual instructions on what changes should be made are provided as an SFTP link in the description of the task. This enables authorized users to upload instructions for task updates to the specific SFTP path. The path needs to be accessible by the CAI SC instance |
The following sample SC_Ops instruction file performs the following operations based on its payload:
-
Retrieves a list of task "watchers" (users who monitor a task's progress) from an SFTP hosted file:
list_item_1_name.json -
Clones the current task for each list_item_1 item (CloneTask)
-
For all cloned tasks, performs operations listed under shared_for_all_clones:
-
Updates the dropdown options for the dropdown attribute named "Dropdown name in CAI workflow" in the original task (setDropDownValue) by retrieving values from an external SFTP hosted file:
list_dd_name.json -
For each cloned task, performs operations listed under per_clone_operations:
-
Updates the "Assignee" with the preset value: ' unassigned@compliance.ai (setAssignee)
-
Updates the "Notify" (task watcher list) with list_item_1 items (setNotify)
{
"SC_Ops": {
"operations": [
[
{
"task_type": "CloneTask",
"list_groups":[{
"mapping":
{
"data":
[
["list_item_name.{'attrib':'list_1_attrib_1', 'scope':'*', 'delim':'-$!-'}"]
]
},
"method": "SFTP:https://complianceai.files.com/path/list_item_1_name.json",
"api_params": [
[
{
"name": "api_method_name",
"value": "results.{'attrib':'attrib_3'}"
}
]
],
"task_type":"list_group"
}],
"shared_for_all_clones": [
{
"task_type": "setDropDownValue",
"mapping":
{
"data":
[
[
"list2_item_name.{'attrib':'attrib_1','scope':'*','delim':'-$!-', 'length':200}"
],
[
"{'pre_val':'-!$-'}"
],
[
"list3_item_name.{'attrib':'attrib_2','scope':'*','delim':'-$!-', 'length':2000}"
],
[
"{'post_proc':'join_attribs','set_delim':'-!$-', 'delim':':', 'split_attribs':'-$!-'}"
]
]
},
"method": "SFTP:https://complianceai.files.com/path/list_dd_name.json",
"name": "Dropdown name in CAI workflow",
"api_params": [
[
{
"name": "api_method_name_2",
"value": "attrib.{'attrib_4':'attrib_4'}"
}
]
]
}
],
"per_clone_operations": [
{
"task_type": "setAssignee",
"mapping":
{
"data":
[
["{'pre_val':'unassigned@compliance__ai'}"]
]
},
"method": "Preset",
"name": "Assignee"
},
{
"task_type": "setNotify",
"mapping":
{
"data":
[
[ "list_groups.items"]
]
},
"method": "Preset",
"name": "Notify"
}
]
}
]
]
}
}
Summary steps to configure and test dynamic updates to Task Information
Connecting CAI SC to 3rd party systems and solutions
Connecting CAI SC to Archer, SAI 360, etc. (SFTP_CSV)
Configurations and mapping of CAI SC and Archer & SAI 360 GRC platform. SC publishes content to an SFTP server, and the content is CSV formatted. These parameters define various settings and mappings between the two systems.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Document_Mapping |
See section Document_Mapping section |
{..} |
|
Annotation_Mapping |
See section Annotation_Mapping section |
{..} |
|
Obligation_Mapping |
See section Obligation_Mapping v |
{..} |
|
Task_Mapping |
See section Task_Mapping section |
{..} |
|
Auth_Mapping_0 |
See section Auth_Mapping_0 section |
{..} |
|
Auth_Mapping_1 |
See section Auth_Mapping_1 section |
{..} |
|
Auth_Mapping_2 |
See section Auth_Mapping_2 section |
{..} |
|
Auth_Mapping_3 |
See section Auth_Mapping_3 section |
{..} |
|
Document_Task_name_prefix |
Prefix for document-related task names |
"CSV_SFTP_DOC" |
String |
Document_Type_Task_name_prefix |
Prefix for document type-related task names |
"AB_DOC" |
String |
Annotation_Task_name_prefix |
Prefix for annotation-related task names |
"AB_ANN" |
String |
Obligation_Relevant_Task_name_prefix |
Prefix for relevant obligation-related task names |
"AB_OBLR" |
String |
Obligation_All_Task_name_prefix |
Prefix for all obligation-related task names |
"AB_OBL" |
String |
Auth_Source_Task_name_prefix |
Prefix for authentication source-related task names |
"AB_RES" |
String |
auth_source_levels |
Level of authentication source hierarchy |
"4" |
String |
Regulations |
Information about regulations |
"" |
String |
ftp_server |
FTP server address |
"complianceai.files.com" |
String |
ftp_uid |
FTP server user ID |
"serviceconnector" |
String |
ftp_pwd |
FTP server password |
"TBD” |
String |
sftp_folder_incremental |
SFTP path for incremental file exports |
"exportfilesCAITEST/incremental" |
String |
sftp_folder |
SFTP Folder path for recent file exports (consolidated) |
"exportfilesCAITEST" |
String |
exportfiles_folder_incremental |
S3 Folder path for incremental file exports |
"exportfilesCAITEST/incremental" |
String |
exportfiles_folder |
S3 Folder path for recent file exports (consolidated) |
"exportfilesCAITEST" |
String |
destination_type |
Type of destination for file transfers |
"sftp" |
String |
doc_name_prefix |
Prefix for document names in the system |
"TEST_Files_" |
String |
recent_document_name |
Filename for the recent document records |
"DOCS_RECENT.csv" |
String |
recent_obligation_name |
Filename for the recent obligation records |
"OBLS_RECENT.csv" |
String |
recent_annotation_name |
Filename for the recent annotation records |
"ANNS_RECENT.csv" |
String |
recent_auth_name_l0 |
Filename for the recent authentication source records (level 0) |
"AUTH_RECENT_L0.csv" |
String |
recent_auth_name_l1 |
Filename for the recent authentication source records (level 1) |
"AUTH_RECENT_L1.csv" |
String |
recent_auth_name_l2 |
Filename for the recent authentication source records (level 2) |
"AUTH_RECENT_L2.csv" |
String |
recent_auth_name_l3 |
Filename for the recent authentication source records (level 3) |
"AUTH_RECENT_L3.csv" |
String |
max_recent_days_records |
Maximum number of recent days' records to retrieve |
"2" |
String |
Connecting CAI SC to MetricStream, 360 Factors, etc. (SFTP)
Configurations and mapping of CAI SC and Archer, MetricStream, 360 Factors GRC platform. SC publishes content to an SFTP server, and the content is JSON formatted. These parameters define various settings and mappings between the two systems.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
Document_Mapping_Strip |
Defines the sections of the document to be included in the mapping. |
["sentence_main", "full_text"] |
Array |
Document_Task_name_prefix |
Prefix for the task name of documents. |
"SFTP_JSON_DOC" |
String |
Annotation_Task_name_prefix |
Prefix for the task name of annotations. |
"SFTP_JSON_ANN" |
String |
Obligation_Relevant_Task_name_prefix |
Prefix for the task name of relevant obligations. |
"SFTP_JSON_OBLR" |
String |
Obligation_All_Task_name_prefix |
Prefix for the task name of all obligations. |
"SFTP_JSON_OBL" |
String |
Auth_Source_Task_name_prefix |
Prefix for the task name of authorization sources. |
"SFTP_JSON_RES" |
String |
auth_source_levels |
Specifies the level of authorization sources to include. |
"0" |
String |
Regulations |
Specifies any relevant regulations. |
"" |
String |
ftp_server |
SFTP server address for file transfer |
"complianceai.files.com" |
String |
ftp_uid |
SFTP username. |
"serviceconnector" |
String |
ftp_pwd |
SFTP password. |
"TODO_FTP_PWD" |
String |
exportfiles_folder_incremental |
Folder path for incremental export files. |
"TODO_TENANT_NAME/incremental" |
String |
Connecting CAI SC to LogicGate (LG)
Configuration and mapping between CAI SC and LogicGate GRC platform. These parameters define various settings and mappings between the two systems.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
LogicGate_wf_regulation_schema |
Mapped to LogicGate Regulations API |
Managed by CAI |
String |
LogicGate_wf_requirement_schema |
Mapped to LogicGate Requirements API |
Managed by CAI |
String |
LogicGate_Mapping_File_Template |
Mapped to LogicGate Mapping -> CAI template (to be used for new installations) |
Managed by CAI |
String |
LogicGate_Mapping_File |
Mapped to LogicGate Regulations API |
Managed by CAI |
String |
LogicGate_Requirement_Mapping_File_Template |
Mapped to LogicGate Requirement Mapping Template (to be used for new installations) |
Managed by CAI |
String |
LogicGate_Requirement_Mapping_File |
Mapped to LogicGate Requirement Mapping file |
Managed by CAI |
String |
LogicGate_Host |
Host address |
https://TODO_TENANT.logicgate.com |
String |
LogicGate_Authorization_URL |
LogicGate Auth URL |
/api/v1/account/token |
String |
LogicGate_Authorization_Code |
LogicGate Auth URL |
Provided by client |
String |
LogicGate_Steps_WF_Regulations_URL |
LogicGate API steps for adding a Regulation |
/api/v1/fields/workflow/TODO_KEY/values |
String |
LogicGate_Steps_WF_Requirement_URL |
LogicGate API steps for adding a Requirement |
/api/v1/fields/workflow/TODO_KEY/values |
String |
LogicGate_Create_Record_URL |
LogicGate API steps for creating a new record |
/api/v1/records/public |
String |
LogicGate_Search_for_Regulation |
LogicGate access to cache & map to Regulations |
/api/v1/records/search?query=CAI_ID&page=0&size=10&operationType=READ&workflow=TODO_KEY |
String |
LogicGate_Search_for_Requirement |
LogicGate access to cache & map to Requirements |
/api/v1/records/search?query=CAI_ID&page=0&size=10&operationType=READ&workflow=TODO_KEY |
String |
LogicGate_link_regulation_with_requirement_URL |
LogicGate access to cache & map Regulatory Requirements |
/api/v1/records/CAI_DOC_ID_TO_LG_REGID/child?layout=1 |
String |
LogicGate_Requirement_Task_name_prefix |
Task prefix to be used in CAI workflow to trigger LogicGate related SC processes |
__LG__ |
String |
Connecting CAI SC to LogicManager (LM)
Configuration and mapping between CAI SC and LogicGate GRC platform. These parameters define various settings and mappings between the two systems.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
"LogicManager_Authorization |
Logic Manager API Authorization code |
Provided by client |
String |
LogicManager_API_Version |
LogicManager API version |
v1_2 |
String |
LogicManager_API_Key |
LogicManager API Key |
Provided by client |
String |
LogicManager_Incidents_URL |
LogicManager API URL for incident management |
https://logicmanager.server.address |
String |
LogicManager_Base |
LogicManager API URL |
ermmarketplace.logicmanager.com |
String |
LogicManager_Host |
LogicManager API Host address |
ermmarketplace.logicmanager.com |
String |
LogicManager_Mapping_File |
Mapping file template |
Managed by CAI |
String |
LogicManager_Reported_by |
Default name of 'Reporter' user in LogicManager |
Provided by client |
String |
LogicManager_Users |
cached copy of LogicManager users |
Provided by client |
String |
LogicManager_Groups |
cached copy of LogicManager groups |
Provided by client |
String |
LogicManager_Preset_Fields |
List of attributes that need to be present in all mapped records |
[ { "key": "Subject", "data": "$cai_category_name$ + Notification: + $doc_title$" }, { "key": "Description", "data": "$cai_category_name$ + Publication: + $doc_title$" }, { "key": "Due Date", "data": "$task_deadline$" } ], |
List |
Connecting CAI SC to AuditBoard Cross Comply (AB)
Configuration and mapping between CAI SC and AuditBoard Cross Comply. These parameters define various settings and mappings between the two systems.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
AuditBoard_Mapping_File |
JSON file for AuditBoard mapping configuration |
Managed by CAI |
String |
AuditBoard_Requirements_File_cached |
Cached CSV file for AuditBoard requirements |
Managed by CAI |
String |
AuditBoard_To_CAI_Mapped_Reqs_cached |
Cached CSV file for AuditBoard to CAI mapped requirements |
Managed by CAI |
String |
AuditBoard_Authorization |
Authorization token for AuditBoard API |
Managed by CAI |
String |
AuditBoard_API_Version |
Version of the AuditBoard API |
v1_2 |
String |
AuditBoard_API_Key |
API key for AuditBoard |
Managed by client (from AuditBoard instance) |
String |
AuditBoard_URL |
URL for accessing AuditBoard framework items API |
https://TODO_TENANT_NAME_.auditboardsupport.com/api/v1/framework_items |
String |
AuditBoard_Base |
Base URL for AuditBoard API |
https://TODO_TENANT_NAME_.auditboardsupport.com/api/v1/ |
String |
AuditBoard_Host |
Host URL for AuditBoard API |
https://TODO_TENANT_NAME_.auditboardsupport.com/api/v1/framework_items |
String |
Docs_To_Publish |
Documents to publish (empty in this case) |
|
String |
bucket_name |
Name of the bucket |
Managed by CAI |
String |
AB_assessment_file |
File for AB assessment |
|
String |
doc_by_section_file |
Document by section file |
|
String |
match_record_attribute |
Attribute for matching records |
field_data.custom_field_pdf_link |
String |
AuditBoard_Preset_Fields |
Preset fields for AuditBoard |
[] |
List |
Document_Mapping |
See section Document_Mapping below |
{..} |
|
Annotation_Mapping |
See section Annotation_Mapping below |
{..} |
|
Attribute_Translate |
See section Attribute_Translate below |
{..} |
|
Attribute_Label_Prefix_Mapping |
See section Attribute_Label_Prefix_Mapping below |
{..} |
|
Document_Task_name_prefix |
Prefix for document task names in the system |
"__AB_DOC__" |
String |
Document_Type_Task_name_prefix |
Prefix for document type task names in the system |
"__AB_DOC__" |
String |
Annotation_Task_name_prefix |
Prefix for annotation task names in the system |
"__AB_ANN__" |
String |
Obligation_Relevant_Task_name_prefix |
Prefix for obligation relevant task names in the system |
"__AB_OBLR__" |
String |
Obligation_All_Task_name_prefix |
Prefix for all obligation task names in the system |
“__AB_OBL__” |
String |
Auth_Source_Task_name_prefix |
Prefix for authentication source task names in the system |
“__AB_RES__” |
String |
Ext_Annot_Assess_File_prefix |
Prefix for external annotation assessment file names |
"__AB_ASSESS_ANNOT_EXT__" |
String |
auth_source_levels |
The number of authentication source levels |
"4" |
String |
Requirements |
|
"" |
|
ftp_server |
The FTP server address |
"complianceai.files.com" |
String |
ftp_uid |
The FTP server username |
Managed by CAI |
String |
ftp_pwd |
The FTP server password (TODO: To be replaced with an actual password) |
Managed by CAI |
String |
exportfiles_folder_incremental |
The folder path for incremental file exports |
Managed by CAI |
String |
exportfiles_folder |
The folder path for file exports |
Managed by CAI |
String |
doc_name_prefix |
Prefix for document names |
"TODO_TENANT_NAME__Files" |
String |
recent_document_name |
The file name for the recent document records |
"DOCS_RECENT.csv" |
String |
recent_obligation_name |
The file name for the recent obligation records |
"OBLS_RECENT.csv" |
String |
recent_annotation_name |
The file name for the recent annotation records |
"ANNS_RECENT.csv" |
String |
recent_auth_name_l0 |
The file name for the recent authentication records at level 0 |
"AUTH_RECENT_L0.csv" |
String |
recent_auth_name_l1 |
The file name for the recent authentication records at level 1 |
"AUTH_RECENT_L1.csv" |
String |
recent_auth_name_l2 |
The file name for the recent authentication records at level 2 |
"AUTH_RECENT_L2.csv" |
String |
recent_auth_name_l3 |
The file name for the recent authentication records at level 3 |
"AUTH_RECENT_L3.csv" |
String |
max_recent_days_records |
The maximum number of days for which recent records are considered |
"2" |
String |
Cache_Org_User_Info |
Flag indicating whether to cache organization user information |
"False" |
Bool |
Cache_Workflow_Info |
|
"False" |
Bool |
Publishing consolidated regulatory changes for the entire organization
CAI SC can consolidate filtered results from various CAI alerts and make them available through our pre-filtered content API end-point . This method supports workflow driven publication, and removes the guessing game in terms of the various cross-organization content filters needed when invoking the CAI API to retrieve content.
In order to configure an CAI SC instance to consolidate results from various alerts into one output, complete the following steps:
-
Setup alerts in CAI Team Edition
-
Setup a CAI workflow and configure all the CAI alerts from the prior step to invoke the newly created CAI workflow
-
Configure the CAI workflow with a CAI SC steps that publishes regulatory sources
-
Configure the CAI SC instance to publish the consolidated responses as active JSON payload
-
The JSON payload includes CAI SC content published within a dynamic window of time, and SC continuously purges older records from the payload
-
Configure CAI SC to maintain the consolidated results in a file named ‘prefiltered.json’:
"Recent_document_name": "pre_filtered_content.json" -
Please note that you can reference the workflow name, alert name and alert owner in the consolidated content to help distinguish which condition (CAI workflow and/or CAI alert) led to the publication of this specific document:
-
"workflow_name": name of the CAI workflow in question
-
"workflow_id": unique id of the CAI workflow in question
-
"alert_name": Name of CAI alert used for this specific content being published
-
"alert_creator_user_name": user id who created the alert that triggered inclusion of this publication
-
Take any necessary actions/complete the required CAI workflow steps (including annotation and labeling)
-
Use the pre-filtered content API to periodically retrieve consolidated results from the end-point.
CAI SC bot: Identification & Authentication
CAI SC user identity is managed using CAI user administration. Each CAI SC is identified as an organization user (either using CAI Team Edition or using SCIM based 3rd party solutions).
CAI SC user authentication supports externally managed (AWS Cognito based) IAM for (SC) bot user accounts.
To instruct an SC instance to use OAUTH based login (instead of CAI managed credential based login), the parameters outlined in the table below need to be configured.
The authentication settings for SC instances are currently managed & maintained by the CAI support team for onboarding, licensing and security reasons.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
service_email |
Email address/mailbox associated with this CAI SC instance. |
Managed by CAI |
String |
service_pwd |
Hashed password stored location for the identity this service uses. |
Managed by CAI |
String |
search_api_key |
API key for the search functionality. This is search API key in Mashery |
Managed by CAI |
String |
auth_method |
Supports using external (AWS Cognito based) login for the Service Connector (SC) bot user |
OAUTH accepted values are: "CAI", "OAUTH". if "CAI" is specified, the legacy credential checking method is used. |
string |
CAI_url_base |
Base URL for the Compliance AI API. |
https://api-production.compliance.ai |
String |
CAI_url_auth_base |
Service Auth configuration |
https://cognito-idp.us-east-1.amazonaws.com/ |
String |
CAI_url_auth_jwt |
Service Auth configuration |
Managed by CAI |
String |
CAI_auth_app_client_id |
Service Auth configuration |
Managed by CAI |
String |
CAI_auth_user_pool_id |
Service Auth configuration |
Managed by CAI |
String |
CAI_non_loggedin_auth_code |
Service Auth configuration |
Managed by CAI |
String |
CAI SC "dot notation" syntax for mapping instructions
CAI SC supports a custom dot notation syntax to retrieve, combine, transform and map specific meta data attributes in CAI content for publication.
Dot notation format
At its simplest form, any CAI attribute can be referenced as a list item (by the name of the CAI attribute as available within the CAI schema).
For example, the following fragment uses the CAI document schema "summary_text"attribute value, and maps it to a user defined key: "description":
CAI.document.summary_text -> External_Schema.description
"description": [
[
"summary_text"
]
]
Dot notation: Concatenating attribute values
The dot notation syntax supports concatenating CAI content attribute values together.
Multiple lists can be combined for a mapped attribute. This feature enables transforming a group of CAI content attributes (mixing multi-value attributes and multiple attributes together) and mapping the resulting content to an external attribute.
For example, in the following fragment, the user-defined attribute "my_agency" is mapped to a concatenated combination of CAI document attributes "agencies.short_name" and "mainstream_news.news_source.name".
The 'dot' between "mainstream_news" , "news_source" and "name" instructs SC to traverse the CAI attribute "mainstream_news" looking for the child element "news_sourc" of the attribute "name".
Similarly, the 'dot' between "agencies" and "short_name" instructs SC to traverse the CAI content attribute "agencies" to identify the child element "short_name" value.
However, the "agencies" attribute in any CAI document could actually include more than one value (multiple agencies may contribute to the publication of the same regulatory document). In order to support multi-value attributes, SC supports the 'scope' indicator, which instructs SC to retrieve all of the values for that attribute. The 'delim' indicator instructs SC to combine the "short_name" attributes using the 'delim' value specified.
{
"my_agency": [
[
"agencies.{'attrib':'short_name' , 'scope':'*' , 'delim':','}"
],
[
"mainstream_news.news_source.name"
]
]
}
Dot notation: Restricting the length of the mapped attribute
User defined destination attributes may have size restrictions, due to data type or other field size constraints.
The 'length' indicator instructs SC to truncate the CAI attribute at the specified number of characters.
In the sample below, the 'title' attribute is cut off at 300 characters before being mapped to 'my_agency'.
{
"my_agency": [
[
title.{'length':300}
],
]
}
Dot notation: Referenced lookup translation
The 'lookup' indicator transforms the content of a CAI attribute based on an external lookup.
The lookup process supports both CAI helper/related content, and user defined lookup processes.
CAI natively supported Lookup include:
-
CAI Document types: DocTypes
-
CAI Jurisdiction: Jurisdictions
-
CAI Regulations: Regulations
-
CAI Acts: Acts
Lookup tables can be retrieved out of band and cached by the CAI SC instance
on a recurring basis.
In the example below, SC extracts the CAI content attribute value for
"cai_category_id" and looks up the corresponding value in the external CAI
'DocTypes' table. By default, 'lookup' assumes a 2 column lookup: SC uses
the value of the CAI attribute as the first column (input) and returns the
value in the second column for that row.
{
"category": [
[
"cai_category_id.{'lookup':'DocTypes'}"
]
]
}
Dot notation: Date format conversion
'Date_convert' converts the format of the CAI attribute to the desired date format.
In the example below, CAI extracts the created_at CAI attribute and maps it to the user defined CreatedDate attribute after converting it to 'UTC' format.
{
"CreatedDate": [
[
"created_at.{'date_convert':'UTC'}"
]
]
}
Dot notation: Using Static content (preset values)
'Pre_val'uses predefined values as part of the mapping.
In the example below, CAI uses the URL specified and concatenates it with the CAI content "id" attribute before mapping it to the user defined pdf_url attribute.
{
"pdf_url": [
[
"{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}"
],
[
"id"
]
]
}
Dot notation: Referencing a specific item within multi-value attributes
[n]. extracts the "nth" item within a multi-valued attribute as part of the mapping.
In the example below, CAI SC performs the following actions:
-
Uses the first list item in the summaries CAI content attribute,
-
Navigates to summary_sentences,
-
Concatenates all the values of that attribute (as specified by the 'scope':'' indicator)
-
Delimiting the items with the characters specified by the 'delim' indicator
-
Truncates the resulting string at 200 characters (as specified by the 'length' indicator)
-
Maps the truncated value to the user-defined "My Auto Summary" attribute.
{
"My Auto Summary": [
[
"summaries.[0].summary_sentences.{'attrib':'*','scope':'*','delim':' ' , 'length':200}"
]
]
}
The following table summarizes the various keywords acceptable as part of the dot notation based attribute mapping instructions.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
scope |
Establishes the treatment for multi valued attributes.
|
'prefix_all'
'*' |
String |
attrib_type |
Treat the attribute content and plain text or as rich text Default: plaintext |
'richtext' 'plaintext' |
String |
main_doc_attrib |
When processing related content types, reference the main document. For example, when mapping to CAI Obligations, cross reference an attribute from associated CAI Document's attribute |
Prefix indicator |
String |
service_var |
Referencing variables from the 'current' document |
Prefix indicator |
String |
lookup |
Referenced lookup for the attribute. Lookup tables need to be pre-defined (pre-referenced) as part of the SC setup. External files can be referenced using the "EXT_assessment" keywords. The name of the external file is expected as the suffix of the EXT_ value. For example to reference lookup from the contents of an file named abc, the value would be EXT_abc |
CAI Document types; DocTypes |
String |
document_task |
Instructs SC to use document task related attributes |
|
String |
'doc_label':'True' |
When processing related content types, reference the main document's labels. For example, when mapping to CAI Annotations, cross references the labels associated with the CAI Document's labels |
Prefix indicator |
String |
date_convert |
Converts the CAI content's date format to another format
Convert to UTC format: UTC |
UTC, ABBR_10 |
String |
attrib |
References the name of an attribute within the CAI content meta data |
title |
String |
delim |
Specifies the delimiter to be used for concatenating the multiple values within a CAI attribute. The delimiter can contain more than one character. |
, |
String |
length |
Truncate the content at the specific characters |
100 |
Integer |
proc_date |
The date CAI SC processed the specific attribute row |
|
String |
wf_name |
The name of the CAI workflow that the SC task is associated with |
Workflow name |
String |
pre_val |
Predetermined value to be used for mapping. If the predetermined value contains dots ("."), each needs to be replaced with 2 underscores ("__") character (to avoid conflict with the dot notation syntax) |
'https://pro__compliance__ai' |
String |
.[n] and .[*] |
Specifies the specific enumerated attribute to use for
multi valued attributes. |
[1] |
List |
truncate |
Dynamic strategy for truncating content (instead of using specific characters) |
CAI Managed |
String |
doc_info |
|
|
|
uplevel |
Instructs the SC to traverse the content upstream "n" levels, looking for the parent of the current document. |
1, 2,.. |
Integer |
alert_name |
The name of the CAI alert that the current document is associated with (caused the workflow to process it) |
alert_1 |
String |
post_proc |
Instructs the SC to perform additional processes after
processing all the other steps specified. |
join_attribs |
String |
split_attribs |
What character to use when splitting a concatenated string. |
',' |
String |
Appendix
Questionnaire
CAI SC questionnaires help gather specific information related to CAI SC mappings, publication and destination infrastructure.. This information is crucial for Compliance.ai to effectively integrate and align with the destination systems document management practices and infrastructure.
GRC Systems Questions
The following information helps us determine the suitable CAI SC configuration for formatting, transforming and publishing regulatory updates.
-
Are you currently using a GRC (Governance, Risk, and Compliance) solution?
-
If yes, identify the GRC from the supported list below.
GRC solution |
Integration Documents |
Protocols |
---|---|---|
Archer |
RSS, SFTP |
|
LogicManager |
API |
|
Galvanize (Diligent) |
RSS |
|
360 Factor |
JSON, SFTP |
|
AuditBoard |
API |
|
LogicGate |
API |
|
ServiceNow |
API |
|
MetricStream |
API/RSS |
|
Navex |
SFTP-CSV |
|
SAI360 |
API |
|
Riskonnect |
API |
-
If not using a supported GRC system:
-
Determine the preferred published content format:
-
Options: CSV, JSON, or XML.
-
Determine the preferred publication method:
-
Option 1- Publish directly to designated content management systems: SFTP, AWS S3, Azure, GDrive, etc. Client to
-
Option 2 - Integrate with customer’s API: Invoke client API & publish to their system
-
Option 3 - Customer retrieves published content using a hybrid model that combines the Service Connector and Compliance.ai API
Data Mapping Questions
-
Make a list of CAI attributes needed: Use Document Search Response Schema to determine required attributes.
-
Provide preferred formatting/structure of the destination attributes (Attributes that surface in the destination system).
Workflow Questions
-
Do you want to review content for relevancy/make annotations prior to them being published?
-
Do you want to review & annotation obligations and/or create annotations on regulatory content before publication?
Sample mapping configuration settings
Sample content mapping: AuditBoard Cross Comply (AB)
Mapping parameters for transforming regulatory documents for usage in AuditBoard Cross Comply application. Specifies how various document fields or attributes are mapped/transformed to externally defined attributes or taxonomies. Each mapping is represented as a key-value pair, where the key is the external (Target) field name and the value is a transformation or extraction rule for the CAI content.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
custom_field_cai_unique_id1 |
Unique ID of the AuditBoard Cross Comply field |
id |
String |
name |
Name of the regulatory framework item in AuditBoard |
title.{'length':300} |
String |
uid |
Unique identifier of the item in AuditBoard |
official_id |
String |
description |
Description of the item |
document_location.{'attrib':'title', 'scope':'*', 'delim':'| '}" |
String |
field_data.custom_field_pdf_link |
Link to the PDF document associated with the item |
{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}, id |
String |
framework_item_custom_multiselect1_option_ids |
AuditBoard Cross Comply Option IDs selected in a custom multi select field |
{'doc_label':'True'}.{'attrib':'label', 'scope':'*', 'delim':',','length':20000}.{'lookup':'RequirementsAB'} |
String |
custom_text3 |
AuditBoard Cross Comply Custom text field |
web_url |
String |
section |
AuditBoard Cross Comply Section of the item |
jurisdiction.{'lookup':'Jurisdiction_parent_jurisdiction_name'} |
String |
subsection |
AuditBoard Cross Comply Subsection of the item |
jurisdiction.{'lookup':'Jurisdiction_name'} |
String |
custom_text5 |
AuditBoard Cross Comply Custom text field |
jurisdiction.{'lookup':'Jurisdiction'} |
String |
custom_text4 |
AuditBoard Cross Comply Custom text field |
cai_category_id.{'lookup':'DocTypes'} |
String |
custom_text6 |
AuditBoard Cross Comply Custom text field |
agencies.{'attrib':'name', 'scope':'*', 'delim':','}, mainstream_news.news_source.name |
String |
field_data.custom_field_text_from_statute_regulation |
AuditBoard Cross Comply Text extracted from the statute or regulation |
full_text.{'length':15000} |
String |
field_data.custom_field_publication_date |
AuditBoard Cross Comply Publication date of the item |
publication_date.{'date_convert':'ABBR_10'} |
String |
field_data.custom_field_effective_date |
AuditBoard Cross Comply Effective date of the item |
rule.effective_on.{'date_convert':'ABBR_10'} |
String |
DateProcessed |
AuditBoard Cross Comply Date when the item was processed |
_PROC_DATE_UTC |
String |
CreatedDate |
AuditBoard Cross Comply Date when the item was created |
DUTC_created_at |
String |
Sample mapping annotations: AuditBoard (AB): Annotation_Mapping
Mapping parameters for transforming user defined annotations within regulatory documents to AuditBoard Cross Comply application . Specifies how various annotation related fields or attributes are mapped/transformed to externally defined attributes or taxonomies. Each mapping is represented as a key-value pair, where the key is the external (Target) field name and the value is a transformation or extraction rule for the CAI content and the associated user-defined annotation.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
See section CAI_matching_conditions below |
{..} |
complex |
|
custom_field_cai_unique_id1 |
Represents a custom field for the unique ID with the value as the ID of the first sentence |
["first_sentence_id"] |
Array |
|
Represents a custom field for the unique ID with a prefix value "_" |
["{'pre_val':'_'}"] |
Array |
|
Represents a custom field for the unique ID with the value as the ID of the last sentence |
["last_sentence_id"] |
Array |
name |
Represents the name parameter with the value as the name extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-name'}"]]} |
Object |
|
Represents the name parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix', 'length':300}"]]} |
Object |
|
Represents the name parameter with the value as the truncated name extracted from the sentences_text |
{"r3": [["sentences_text.{'truncate':'AB_name','length':300}"]]} |
Object |
subsection |
Represents the subsection parameter with the value as the subsection extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-subsection'}"]]} |
Object |
|
Represents the subsection parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix', 'length':300}"]]} |
Object |
|
Represents the subsection parameter with the value as the truncated label extracted from the labels |
{"r3": [["labels.{'attrib':'label', 'scope':'0', 'lookup':'doc_by_section-Grouped_Sentence', 'length':300, 'truncate':'AB_subsection'}"]]} |
Object |
custom_text4 |
Represents a custom text parameter with the value as the custom_text4 extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-custom_text4'}"]]} |
Object |
|
Represents a custom text parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix', 'length':300}"]]} |
Object |
framework_item_custom_select5_option_id |
Represents the framework_item_custom_select5_option_id parameter with the value as the option ID extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-framework_item_custom_select5_option_id'}"]]} |
Object |
|
Represents the framework_item_custom_select5_option_id parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix', 'length':300}"]]} |
Object |
uid |
Represents the UID parameter with the value as the UID extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-uid'}"]]} |
Object |
|
Represents the UID parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix','length':2000}"]]} |
Object |
|
Represents the UID parameter with the value as the label extracted from the labels without a prefix scope |
{"r3": [["labels.{'attrib':'label', 'scope':'no_prefix','length':2000}"]]} |
Object |
|
Represents the UID parameter with the value as a combination of doc_info.id, first_sentence_id, and last_sentence_id with prefix values "_" |
{"r4": [["doc_info.id"]], ["{'pre_val':''}"], ["first_sentence_id"], ["{'pre_val':''}"], ["last_sentence_id"]]} |
Object |
custom_text5 |
Represents a custom text parameter with the value as the jurisdiction extracted from doc_info |
[["doc_info.jurisdiction.{'lookup':'Jurisdiction'}"]] |
Array |
custom_text6 |
Represents a custom text parameter with the value as the agencies' names extracted from doc_info and the mainstream news source's name |
[["doc_info.agencies.{'attrib':'name', 'scope':'*', 'delim':','}"], ["doc_info.mainstream_news.news_source.name"]] |
Array |
description |
Represents the description parameter with the value as the sentences_text |
[["sentences_text"]] |
Array |
custom_text3 |
Represents a custom text parameter with the value as the web_url extracted from doc_info |
[["doc_info.web_url"]] |
Array |
section |
Represents the section parameter with the value as the section extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-section'}"]]} |
Object |
|
Represents the section parameter with the value as the label extracted from doc_info with a prefix scope |
{"r2": [["doc_info.{'doc_label':'True'}.{'attrib':'label','scope':'prefix','length':300}"]]} |
Object |
|
Represents the section parameter with the value as the title extracted from doc_info |
{"r3": [["doc_info.title.{'length':300}"]]} |
Object |
framework_item_custom_select2_option_id |
Represents the framework_item_custom_select2_option_id parameter with the value as a prefix value "4" |
[["{'pre_val':'4'}"]] |
Array |
field_data.custom_field_pdf_link |
Represents the custom field for a PDF link with the value as a combination of prefix values, doc_info.id, first_sentence_id, and last_sentence_id |
[["{'pre_val':'https://pro__compliance__ai/content?overlay=pdf-overlay&renderfromtask=tasks&summary_id='}"], ["doc_info.id"], ["{'pre_val':'&ag='}"], ["first_sentence_id"], ["{'pre_val':'_'}"], ["last_sentence_id"]] |
Array |
framework_item_custom_select4_option_id |
Represents the framework_item_custom_select4_option_id parameter with the value as the option ID extracted from the first sentence |
{"r1": [["first_sentence_id.{'lookup':'EXT_assessment-framework_item_custom_select4_option_id'}"]]} |
Object |
|
Represents the framework_item_custom_select4_option_id parameter with the value as the label extracted from the labels with a prefix scope |
{"r2": [["labels.{'attrib':'label', 'scope':'prefix','length':2000}"]]} |
Object |
Comments |
Represents the Comments parameter with the value as the comments extracted from the comment_threads |
[["comment_threads.[].comments.{'attrib':'richtext', 'scope':'', 'delim':',', 'l':20000, 'attrib_type':'richtext'}"]] |
Array |
Mapping CAI labels to AuditBoard: Attribute_Label_Prefix_Mapping
The "Attribute_Label_Prefix_Mapping" in the provided JSON represents a mapping between attributes or labels used in AuditBoard and their corresponding prefixes in Compliance.ai service connector. This mapping helps in aligning and matching the attributes or labels between the two systems.
These mappings facilitate the matching and synchronization of attributes and labels between AuditBoard and Compliance.ai service connector.
For example, when transferring data or information from AuditBoard to Compliance.ai, the attribute "uid" in AuditBoard will be mapped to its corresponding prefix "uid" in Compliance.ai. Similarly, the attributes "section," "subsection," and "name" will be mapped to their respective prefixes. The label prefixes such as "reg applicability," "country," "lifecycle," and "Cons/FinCrime" provide context or categorization to the associated attributes or labels in Compliance.ai.
Parameter-Target |
Description |
Value-Source |
Type |
---|---|---|---|
uid |
The unique identifier attribute in AuditBoard |
"uid" |
String |
section |
The section attribute in AuditBoard |
"section" |
String |
subsection |
The subsection attribute in AuditBoard |
"subsection" |
String |
name |
The name attribute in AuditBoard |
"name" |
String |
framework_item_custom_select5_option_id |
The label prefix for regulatory applicability |
"reg applicability" |
String |
framework_item_custom_select2_option_id |
The label prefix for country |
"country" |
String |
custom_text4 |
The label prefix for lifecycle |
"lifecycle" |
String |
framework_item_custom_select4_option_id |
The label prefix for consolidated/financial crime |
"Cons/FinCrime" |
String |