Wiki source code of Utilities - DExTr

Last modified by rayaan@one-count_com on 2023/11/09 10:38

Show last authors
1 {{layout}}
2 {{layout-section ac:type="two_right_sidebar"}}
3 {{layout-cell}}
4
5
6 {{id name="Start ScreenSteps Content"/}}
7
8 DExTr is ONEcount’s data exchange and transformation engine. This tool enables data exchanges between any data source or third-party vendor. Establish a data map once and, going forward, easily import and export data in and out of ONEcount’s Customer Data Platform.
9
10 This tool the capabilities to map and sync first- or third-party data to ONEcount questions/data fields.
11
12 If there are multiple databases with similar fields, DExTr can consolidate the similar datapoints into a single ONEcount question or keep the fields separate. //(Example: if the field for ‘Customer ID’ in a legacy database is “CUST_ID” and the field name in a CRM is “CID”, there is an option to have the data consolidated in to the single ONEcount question that may be named “CustomerID”.//
13
14 **If you will be using the Dashboard Feed Report function to generate an automated feed report or will be performing a data import that requires mapping external file questions/data to ONEcount questions/data, you must complete the report mapping here first.**
15
16 There are three types of mapping in the Mapper: Demographic Mapping and Transaction Mapping.
17
18 * (((
19 **Demographic Mapping:** Demographic mapping maps external demographic questions/data fields and responses to ONEcount demographic questions and responses.
20 )))
21 * (((
22 **Transaction Mapping:** Transaction mapping maps external transaction data, e.g. product purchase activity, status, and date formats.
23 )))
24 * (((
25 **Engagement Mapping:** Activity mapping maps external activity headers, e.g. time on page, URL, impressions, etc. This mapping is used primarily for feed reports for export, or importing activity from another system like an e-mail system.
26 )))
27
28 One or all of these types of mapping may be required, depending on the type of automated feed report you want to generate or the data import you'll be performing.
29
30 You can either import a layout file (.csv format) with all of the necessary demographic, transacactional and engagement fields,  and map demographics and transactions directly from that file, or you can manually enter the demographics, transactions, and activities you wish to map to ONEcount questions/data fields, or do a combination of both.
31
32 **Please Note:**
33
34 **ONEcount questions, response valuess, products and packages must be created prior to performing mapping.**
35
36 **Third-party activity headers currently can't be mapped via an uploaded layout file. They must be mapped manually in the Activity Mapping Area.**
37
38 **OPEN THE MAPPING WINDOW**
39
40 Select DATA MANAGEMENT from the main menu and the DExTr from the sub-menu.
41
42 [[image:attach:Screen Shot 2021-10-09 at 4.33.36 PM (2) copy 8.png||thumbnail="true" height="250"]]
43
44 This will open the Main DExTr Listing Page, as shown below.
45
46 **CREATE A NEW DExTr MAPPING**
47
48 [[image:attach:DExTr Listing Page.png||height="250"]]
49
50 \\
51
52 Select "Create Mapping" to create a new mapping. This will open the Mapping Builder, as shown below.
53
54 To edit an existing mapping, select the Edit icon.
55 To copy an existing mapping, select the Copy icon.
56 To delete an existing mapping, select the Delete icon.
57 \\
58
59 \\
60
61 (% style="margin-left: 30.0px;" %)
62 \\
63
64 === Creating a Data Mapping ===
65
66 **[[image:attach:New-DExTr-Mapping.png||height="250"]]**
67
68 \\
69
70 As you can see, a DExTr mapping consists of the following parts. You may use any or all of these components. The only required component to save the mapping is the mapping Title. All other components are optional.
71
72 Navigate across the six tabs at the top of the interface to configure how the customer data platform will interpret and exchange data.
73
74 1. (((
75 (((
76 Exchange Map** - **Define Exchange Map
77 )))
78
79 (((
80 1. (((
81 Enter a name and description for this data exchange, then proceed to the next tab (Demographic Fields).
82 )))
83 )))
84 )))
85
86 \\
87
88 [[image:attach:DExTr Demo Tab.png||width="800"]]
89
90 \\
91
92 1. (((
93 Demographic Fields** - ** Map external demographic questions/data fields and responses to ONEcount demographic questions and responses.
94 )))
95
96 Define field names (the column headers from the import file) on the left and match them to the ONEcount field on the right.
97
98 If the ONEcount field does not yet exist, you'll need to create it using the Question Builder (see “Getting Started” section above).Map Values & Field Response Labels (non-text fields only)
99
100 If the response is not a text field and is either a drop down, radio button or check box selection, click the [ Map Values] button in the row under the “Map Responses” column. The Map Values button will open a pop-up window. Listed on the left are ONEcount Response Labels (and values). These are the responses defined for the ONEcount Question paired to the field from the database.  On the right, enter the values that will appear in the import file.  It is very important to map all values, because if a value in the input file is not found in ONEcount, the response for that field will be dropped. ONEcount will not allow external imports to add values to its response table. Also, if the values in your import file exactly match the values that ONEcount has (ie., for the State question, both the import/export file and ONEcount use the two-letter abbreviations) then you do not need to create a mapping.
101
102 \\
103
104 [[image:attach:DExTr Value Mapping Pop-Up.png||height="250"]]
105
106 \\
107
108 \\
109
110 Define how ONEcount should translate each response using the fields in the right column, ‘Field Response’. DExTr can use the same responses, or you may choose to translate or ‘transform’ data to another format.
111
112 //(Example: states may be abbreviated with 4 letters in a POS database [Conn., Mass.], while the ecommerce database has two letter abbreviations.)//
113
114 \\
115
116 Optional: add validations for the field to either check for duplicates, require an entry, or verify the format entered to the field.Click “Add” to enter additional demographic fields.
117
118 1. (((
119 Transactional Fields** - **Map external transaction data, e.g. product activity, status, and date formats.
120 )))
121
122 * (((
123 1. (((
124 Define transactional fields by entering the field name from the import file, the field type and then match the value of the field to the value in ONEcount by clicking the [Map] button.
125 )))
126 1. (((
127 i.e. if the source/import file has a product field called “PUB_TITLE” and the value for current subscribers is “YES” on the import file, but the ONEcount product is “Active Subscriber”, the Package Status Value from the import file should be matched with the Package Status in ONEcount.)
128 )))
129 1. (((
130 Within the “Map Values” pop up window, click “Add” to define all possible transactional statuses.
131 )))
132
133 1.
134
135 Engagement Fields** - **Map external engagement (activity) headers, e.g. time on page, URL, impressions, email opens, registrations, link clicks, etc. Engagement field mapping is used primarily for exporting feed reports.
136 )))
137
138 * (((
139 1. (((
140 To map engagement values of fields, define the engagement field name (the column header from the import file) and match it to the engagement type in ONEcount.
141 )))
142 1. (((
143 If the Engagement Field Value is a metric that is not a count, click the [] icon to map the metric field value to the ONEcount metric
144
145 //(i.e. if the source/import file has an ‘Engagement Field Name’ that is “NEWSLETTER_ACTIVITY_TYPE” the ‘ONEcount Engagement’ would be “Metric >> Activity Metric Name” and then map the values of metric type [“OPEN”, “CLICK”] to the ONEcount metric [“Email opens”/”Email clicks”]). //
146 )))
147 )))
148
149 5.    Static Fields
150
151 1. (((
152 *Optional Step: If a column and values needs to be added for identification, record keeping, sourcing, etc. purposes, DExTr can append data to each row.
153 )))
154 1. (((
155 Define the “Static Field Name” which will be the column heading on the export file (i.e. “DATA_SYNC_NAME”), and then the value of the static field (i.e. “CRM_SYNC”).
156 )))
157 1. (((
158 Click “Add” to create additional static field appends.
159 )))
160
161 6.    Re-order Fields
162
163 1. (((
164 *Optional Step: If the order of the columns needs to be re-arranged for the order in which they appear on the export from a data feed: simply drag and drop the order of fields by clicking and holding the field labels.
165 )))
166
167 \\
168
169 \\
170
171 Transaction Mapping is done the same way as Demographic Mapping by using the drag and drop functionality to match Third-Party transaction values with ONEcount transaction values.
172
173 There are four Mapping Areas for Transaction Mapping:
174
175 * (((
176 Product Mapping Area
177 )))
178 * (((
179 Product Status Mapping Area (optional)
180 )))
181 * (((
182 Subscription Mapping Area
183 )))
184 * (((
185 Date Mapping Area
186 )))
187
188 **NOTE THAT MAPPING IS ID TO ID IN TRANSACTION MAPPING. THE ID IS FOUND IN PARENTHESIS AFTER THE TEXT IN THE "THIRDPARTY..." COLUMN OF MAPPING AREAS.**
189
190 **AS WITH DEMOGRPHIC MAPPING, ONECOUNT PRODUCTS, PRODUCT STATUSES, AND SUBSCRIPTION TYPES MUST BE CREATED PRIOR TO PERFORMING TRANSACTION MAPPING.**
191
192 **THIRD-PARTY TRANSACTION VALUES CAN BE PRE-POPULATED BY UPLOADING A LAYOUT FILE OR MANUALLY ENTERED (AS DESCRIBED IN "INFORMATION" SECTION ABOVE).**
193
194 *(% style="text-decoration: underline;" %)Product Mapping Area(%%) *
195 **For automated data feed reports**, the Third Party Product column shows what the ONEcount product name will be displayed as on output.
196
197 * (((
198 For example, a user is subscribed to/registered for ABCXY Pharma Aug. Webinar(207) in ONEcount. The third party requires that on the report output show a value of August Webinar rather than ABCXY Pharma Aug. Webinar. Mapping ONEcount's ABCXY Pharma Aug. Webinar(207) product to August Webinar(400) will display the Third Party Product value of August Webinar on the output.
199
200 **Product Column Field:** This identifies what to output as the header for the product field. Enter the heading that's required for the report output.
201 )))
202
203 **For imports,** the Third Party Product column identifies the names/values from the import file that will be mapped to the ONEcount product.
204
205 * (((
206 For example, there's "product" column in the import file that identifies what product a particular record is subscribed to or registered for. In the import file, August Webinar in that colum indicates the record is subscribed to/registered for the August Webinar. In ONEcount, the product name is ABCXY Pharma Aug. Webinar. Any record in the import file that has August Webinar in the product column will be subscribed to the ABCYXY Pharma Aug. Webinar product in ONEcount.
207
208 **Product Column Field:** This identifies the column header for the product column in the import file. In the above example, you would enter "product".
209 )))
210
211 (% style="text-decoration: underline;" %)**Product Status Mapping Area (Optional)**(%%)
212 **For automated data feed reports,** the Third Party Product Status column shows what the ONEcount product status will be displayed as on output.
213
214 * (((
215 For example, a user has a product status of "Not Active(405)" in ONEcount. The third party requires that the report output show a value of Inactive, rather than Not Active. Mapping ONEcont's "Not Active(405)" status to "Inactive(203)" will display the Third Party Product Status of Inactive on the output.
216
217 **Product Status Column Field:** This identifies what to output as the header for the product status field. Enter the heading that's required for the report output.
218 )))
219
220 **For Imports**, the Third Party Product Status column identifies what name/value from the import file will be mapped to the ONEcount product.
221
222 * (((
223 For example, there's a "product status" column in the import file that identifies the product status a particular record has for a particular product. In the import file, "Inactive(203)"in that column indicates the record has a product status of inactive. In ONEcount, the product status is "Not Active(405)". Any record in the import file that has a product status of "Inactive" in the product status column will have the product status of "Not Active" applied to the record in ONEcount.
224
225 **Product Status Column Field:** This identifies the column header for the product status column in the import file. In the above example, you would enter "product status".
226 )))
227
228 (% style="text-decoration: underline;" %)**Subscription Mapping Area**(%%)
229 **For automated data feed reports,** the Third Party Subscription column shows what the ONEcount subscription type will be displayed as on output.
230
231 * (((
232 For example, a user has a subscription type of "n" in ONEcount. The third party requires that the report output show a value of new, rather than "n". Mapping ONEcont's "n" status to "new(new)" will display the Third Party Product Status of new on the output.
233
234 **Subscription Column Field:** This identifies what to output as the header for the subscription type field. Enter the heading that's required for the report output.
235 )))
236
237 **For Imports**, the Third Party Subscription column identifies what name/value from the import file will be mapped to the ONEcount product.
238
239 * (((
240 For example, there's a "Subcription Type" column in the import file that identifies the subscription type a particular record has for a particular product. In the import file, "New" in that column indicates the record has a subscription type of new. In ONEcount, the subscription type is "n". Any record in the import file that has a product status of "New" in the subscription type column will have the product status of "n" applied to the record in ONEcount.
241
242 **Subscription Column Field:** This identifies the column header for the subscription type column in the import file. In the above example, you would enter "Subscription Type".
243 )))
244
245 (% style="text-decoration: underline;" %)**Date Mapping Area**(%%)
246 **For automated data feed reports,** enter the format you would like dates to be exported as. For example MM/dd/yyyy will export as 07/05/2023 and yy/MM/dd will export as 23/07/05. If you want to import time along with date add hh:mm:ss:a along with date. For example MM/dd/yyyy hh:mm:ss:a will be exported as 07/23/2023 11:45:32:AM
247
248 (% style="color: rgb(51,51,51);" %)MM : months, dd: days, yyyy: years, hh: hours (12 hour format), HH: hours (24 hour format), mm: minutes, ss: seconds, a:AM/PM(%%)\\
249
250 **Date Column Field:** This identifies what to output as the header for the date field. Enter the heading that's required for the report output.
251
252 **For Imports,** enter the format of dates in the import file. For example, if a date is 07/05/2016, enter the format MM/dd/yyyy. If you want to import time along with date add hh:mm:ss along with date. For example MM/dd/yyyy hh:mm:ss:a will be exported as 07/23/2023 11:45:32:AM
253
254 (% style="color: rgb(51,51,51);" %)MM : months, dd: days, yyyy: years, hh: hours (12 hour format), HH: hours (24 hour format), mm: minutes, ss: seconds, a:AM/PM
255
256 **Date Column Field:** This identifies the column header for the date column in the import file.
257
258 When done with mapping, click "Next".
259 You will then be brought either to Activity Mapping or Save, depending on selection made in 1 – Information.**SAVE MAPPING**
260
261 When done with mapping(s), click "Next".
262
263 You then have the option to save the mapping (Save) or return to mapping (Previous).
264
265 Once mapping is saved, you can find it on the Manage Mapping listing page.
266
267 A saved mapping is ready to be used for generating automated feed reports in the Dashboard. See Dashboard Reports – Create Feed Report for more information.
268
269
270
271 {{id name="End ScreenSteps Content"/}}
272
273 === MAPPING ENGAGEMENT DATA ===
274
275 This section outlines the types of engagement data that DExTr can import/export. Please note that Generic and Engagement data fields can be used for both web, banner and newsletter engagement data.
276
277 If you need data points imported or exported that are not on this list, check with the ONEcount team to see if those fields can be accommodated.
278
279 Generic Engagement Fields
280
281 * Activity Date: Date of the Activity
282 * Activity Time: Time of the Activity, Mandatory for Import.
283 * Channel Codes: Codes for Email, Banner and Web page views, CMI needed it in export.
284 * End Date: End date of export, mainly used in aggregate export.
285 * Unique External ID: Unique ID for each transaction.
286 * Month: Month of the Activity.
287 * OCID:
288 * OCID Hash: Unique hash of the user. This is the cookie set on user’s browser.
289 * Platform Viewed: Platform of the user in which Impression or Page view happened.
290 * Resource Details: This is used for export. It has Page title for Web activity, Banner name for Banner Activity and Message title for Email data.
291 * Start Date: Start date of export, mainly used in aggregate export.
292
293 Metric Engagement Fields
294
295 * Activity Metric code: Metric codes for Activity data. It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view. It is mandatory for Import and export.
296 * Activity Metric Name: Metric Names for Activity data (CMI report needs Metric code and Name). It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view.
297 * Activity Metric count: Mainly used for export. It has counts for each activity.
298 * Aggregate Metric code: Metric codes for Aggregate data. It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view. Mandatory for aggregate export.
299 * Aggregate Metric Name: Metric Names for Aggregate data (CMI report needs Metric code and Name). It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view.
300 * Aggregate Metric count: Mainly used for export. It has counts for each activity.
301
302 Banner Engagement
303
304 * Ad Server Name: Name of the Ad Server, i.e., DFP
305 * Advertiser ID: ID of the Advertiser
306 * Advertiser Name: Name of the Advertiser
307 * Banner ID: ID of the Banner
308 * Banner Name: Name of the Banner
309 * Campaign ID: ID of the Campaign
310 * Campaign Name: Name of the Campaign
311 * Viewable Impression: If Impression is viewed or not, Boolean (0, 1 accepted)
312
313 Newsletter Engagement Fields
314
315 * Click URL: URL of the Clicked link. Mandatory for Importing Email clicks.
316 * List ID: ID of the newsletter List of Group
317 * List Name: Name of the newsletter list or group
318 * Message ID: Campaign ID of the newsletter
319 * Message Name: Campaign name of the newsletter
320 * Recipient ID: Unique User Id in the Email service provider
321 * Message Subject: Subject of the Campaign
322
323 Web Engagement Fields
324
325 * Referrer: Referrer for the page view
326 * Site name: Website URL
327 * Page URL: URL of the Page.
328
329 === **HOW DExTr MAPS DATA** ===
330
331 The following information is being provided to help you understand how DExTr will map/transform your data on import.
332
333 (% class="diff-block-target" %)
334 1. On import, ONEcount will always check if fields in the DExTr mapping are present in the file or not; if any field is not present in the file, then we fail the job. This is to ensure that we won’t miss mandatory data or dedupe fields and possibly corrupt the data. If you are importing data and the file does not have all of the fields in the DExTr mapping you are using, you can create a blank column with that field name in it–//as long as that field is not a de-dupe field for the import.//
335 1. If there are multiple dedupe fields in mapping then we dedupe the data based on order that the fields appear in the mapping. For Example if mapping has dedupe fields as “Email” (1^^st^^ field in order) and “AccountID” (2^^nd^^ field in order) then DExTr will first query the datase for a match on “Email,” if match is not found then it will query on “AccountID”. It always an OR match. You //can not match// on Email AND AccountID
336 1. If OCID is present as a dedupe field, then it always takes precedence irrespective of order. If you import a file with OCID as a dedupe field, and a record does not have an OCID, then a new user will be created in ONEcount and a unique OCID will be generated.
337 1. There are a variety of validations which can be added to DExTr mapping like, “Numeric”, “Required” etc.. If any record/row doesn’t satisfy these validations, then that record won’t be imported into ONEcount. It will be saved into a failed record file and mailed to the email address supplied with the import job.
338 1. If there are any passwords that are being imported, they should be imported as plain text. ONEcount will encrypt the field based on encrypt function for your installation.
339 1. For Package transactions: Term ID, Package status, source code and transaction type are mandatory.
340 11. If Term ID is not mapped, then the transaction won’t be imported.
341 11. IF Package status is not mapped then default Package status from term will be added.
342 11. If source code is not mapped. Then default source code from import job will be selected.
343 11. If transaction type is not added, then default type from Import job will be selected.
344 1. For Product transaction: Product ID, source code and transaction type are mandatory.
345 11. If Product ID is not mapped, then the transaction won’t be imported.
346 11. If source code is not mapped. Then default source code from import job will be selected.
347 11. If transaction type is not added, then default type from Import job will be selected.
348 1. If Transaction Date is not mapped or not formatted correctly, then current date will be used as request date.
349 1. If Expiration date is not mapped or not formatted correctly, then expiration date will be calculated based on the duration of the term and the request date.
350 1. For Engagement imports, Activity type is mandatory.
351 1. If Engagement date is not formatted, then current date will be used as activity date.
352 1. If Data and campaign ID, name, List ID and name then they will be imported as well.
353 1. If dedupe is not added/matched for an engagement import then new users will be created in the system just like Users and transaction import.
354
355 \\
356 {{/layout-cell}}
357
358 {{layout-cell}}
359 {{panel borderStyle="solid" title="Search all Documentation:"}}
360
361
362 {{livesearch/}}
363 {{/panel}}
364
365 {{panel borderStyle="solid" title="Search System UTILITIES:"}}
366
367
368 {{pagetreesearch rootPage="System Utilities"/}}
369 {{/panel}}
370
371 {{panel title="More System UTILITIES"}}
372 {{expand title="Expand"}}
373
374
375 (% style="margin-left: 30.0px;" %)
376 [[Job List>>doc:Utilities - Job List]]
377
378 (% style="margin-left: 30.0px;" %)
379 [[Backend User Permissions>>doc:Utilities - Backend Users and Permissions]]
380
381 (% style="margin-left: 30.0px;" %)
382 [[Email Engine Sender Score>>doc:ONEmail and Sender Score]]
383
384 (% style="margin-left: 30.0px;" %)
385 [[doc:ONEmail]]
386
387 (% style="margin-left: 30.0px;" %)
388 Action Manager
389
390 (% style="margin-left: 30.0px;" %)
391 [[Configuration Manager>>doc:Utilities - Configuration Manager]]
392
393 (% style="margin-left: 30.0px;" %)
394 Backend Activity Log
395
396 (% style="margin-left: 30.0px;" %)
397 IVT Mitigator
398
399 (% style="margin-left: 30.0px;" %)
400 [[Widget Builder>>doc:Utilities - Widget Builder]]
401
402 (% style="margin-left: 30.0px;" %)
403
404
405 (% style="margin-left: 30.0px;" %)
406 Import/Export Tool
407 {{/expand}}
408 {{/panel}}
409
410 \\
411
412 \\
413 {{/layout-cell}}
414 {{/layout-section}}
415 {{/layout}}