Wiki source code of Utilities - DExTr

Version 18.1 by Admin User on 2025/04/30 09:35

Show last authors
1 {{layout}}
2 {{layout-section ac:type="two_right_sidebar"}}
3 {{layout-cell}}
4 {{id name="Start ScreenSteps Content"/}}
5
6 DExTr is ONEcount’s data exchange and transformation engine. This tool enables data exchanges between any data source or third-party vendor. Establish a data map once and, going forward, easily import and export data in and out of ONEcount’s Customer Data Platform.
7
8 This tool the capabilities to map and sync first- or third-party data to ONEcount questions/data fields.
9
10 If there are multiple databases with similar fields, DExTr can consolidate the similar datapoints into a single ONEcount question or keep the fields separate. //(Example: if the field for ‘Customer ID’ in a legacy database is “CUST_ID” and the field name in a CRM is “CID”, there is an option to have the data consolidated in to the single ONEcount question that may be named “CustomerID”.//
11
12 **If you will be using the Dashboard Feed Report function to generate an automated feed report or will be performing a data import that requires mapping external file questions/data to ONEcount questions/data, you must complete the report mapping here first.**
13
14 There are three types of mapping in the Mapper: Demographic Mapping and Transaction Mapping.
15
16 * (((
17 **Demographic Mapping:** Demographic mapping maps external demographic questions/data fields and responses to ONEcount demographic questions and responses.
18 )))
19 * (((
20 **Transaction Mapping:** Transaction mapping maps external transaction data, e.g. product purchase activity, status, and date formats.
21 )))
22 * (((
23 **Engagement Mapping:** Activity mapping maps external activity headers, e.g. time on page, URL, impressions, etc. This mapping is used primarily for feed reports for export, or importing activity from another system like an e-mail system.
24 )))
25
26 One or all of these types of mapping may be required, depending on the type of automated feed report you want to generate or the data import you'll be performing.
27
28 You can either import a layout file (.csv format) with all of the necessary demographic, transacactional and engagement fields,  and map demographics and transactions directly from that file, or you can manually enter the demographics, transactions, and activities you wish to map to ONEcount questions/data fields, or do a combination of both.
29
30 **Please Note:**
31
32 **ONEcount questions, response valuess, products and packages must be created prior to performing mapping.**
33
34 **Third-party activity headers currently can't be mapped via an uploaded layout file. They must be mapped manually in the Activity Mapping Area.**
35
36 **OPEN THE MAPPING WINDOW**
37
38 Select DATA MANAGEMENT from the main menu and the DExTr from the sub-menu.
39
40 [[image:attach:Screen Shot 2021-10-09 at 4.33.36 PM (2) copy 8.png||thumbnail="true" height="250"]]
41
42 This will open the Main DExTr Listing Page, as shown below.
43
44 **CREATE A NEW DExTr MAPPING**
45
46 [[image:attach:DExTr Listing Page.png||height="250"]]
47
48
49 Select "Create Mapping" to create a new mapping. This will open the Mapping Builder, as shown below.
50
51 To edit an existing mapping, select the Edit icon.
52 To copy an existing mapping, select the Copy icon.
53 To delete an existing mapping, select the Delete icon.
54
55
56
57
58 === Creating a Data Mapping ===
59
60 **[[image:attach:New-DExTr-Mapping.png||height="250"]]**
61
62
63 As you can see, a DExTr mapping consists of the following parts. You may use any or all of these components. The only required component to save the mapping is the mapping Title. All other components are optional.
64
65 Navigate across the six tabs at the top of the interface to configure how the customer data platform will interpret and exchange data.
66
67 1. (((
68 (((
69 Exchange Map** - **Define Exchange Map
70 )))
71
72 (((
73 1. (((
74 Enter a name and description for this data exchange, then proceed to the next tab (Demographic Fields).
75 )))
76 )))
77 )))
78
79 [[image:attach:DExTr Demo Tab.png||width="800"]]
80
81
82 1. (((
83 Demographic Fields** - ** Map external demographic questions/data fields and responses to ONEcount demographic questions and responses.
84 )))
85
86 Define field names (the column headers from the import file) on the left and match them to the ONEcount field on the right.
87
88 If the ONEcount field does not yet exist, you'll need to create it using the Question Builder (see “Getting Started” section above).Map Values & Field Response Labels (non-text fields only)
89
90 If the response is not a text field and is either a drop down, radio button or check box selection, click the [ Map Values] button in the row under the “Map Responses” column. The Map Values button will open a pop-up window. Listed on the left are ONEcount Response Labels (and values). These are the responses defined for the ONEcount Question paired to the field from the database.  On the right, enter the values that will appear in the import file.  It is very important to map all values, because if a value in the input file is not found in ONEcount, the response for that field will be dropped. ONEcount will not allow external imports to add values to its response table. Also, if the values in your import file exactly match the values that ONEcount has (ie., for the State question, both the import/export file and ONEcount use the two-letter abbreviations) then you do not need to create a mapping.
91
92
93 [[image:attach:DExTr Value Mapping Pop-Up.png||height="250"]]
94
95
96
97 Define how ONEcount should translate each response using the fields in the right column, ‘Field Response’. DExTr can use the same responses, or you may choose to translate or ‘transform’ data to another format.
98
99 //(Example: states may be abbreviated with 4 letters in a POS database [Conn., Mass.], while the ecommerce database has two letter abbreviations.)//
100
101
102 Optional: add validations for the field to either check for duplicates, require an entry, or verify the format entered to the field.Click “Add” to enter additional demographic fields.
103
104 1. (((
105 Transactional Fields** - **Map external transaction data, e.g. product activity, status, and date formats.
106 )))
107
108 * (((
109 1. (((
110 Define transactional fields by entering the field name from the import file, the field type and then match the value of the field to the value in ONEcount by clicking the [Map] button.
111 )))
112 1. (((
113 i.e. if the source/import file has a product field called “PUB_TITLE” and the value for current subscribers is “YES” on the import file, but the ONEcount product is “Active Subscriber”, the Package Status Value from the import file should be matched with the Package Status in ONEcount.)
114 )))
115 1. (((
116 Within the “Map Values” pop up window, click “Add” to define all possible transactional statuses.
117 )))
118
119 1.
120
121 Engagement Fields** - **Map external engagement (activity) headers, e.g. time on page, URL, impressions, email opens, registrations, link clicks, etc. Engagement field mapping is used primarily for exporting feed reports.
122 )))
123
124 * (((
125 1. (((
126 To map engagement values of fields, define the engagement field name (the column header from the import file) and match it to the engagement type in ONEcount.
127 )))
128 1. (((
129 If the Engagement Field Value is a metric that is not a count, click the [] icon to map the metric field value to the ONEcount metric
130
131 //(i.e. if the source/import file has an ‘Engagement Field Name’ that is “NEWSLETTER_ACTIVITY_TYPE” the ‘ONEcount Engagement’ would be “Metric >> Activity Metric Name” and then map the values of metric type [“OPEN”, “CLICK”] to the ONEcount metric [“Email opens”/”Email clicks”]). //
132 )))
133 )))
134
135 5.    Static Fields
136
137 1. (((
138 *Optional Step: If a column and values needs to be added for identification, record keeping, sourcing, etc. purposes, DExTr can append data to each row.
139 )))
140 1. (((
141 Define the “Static Field Name” which will be the column heading on the export file (i.e. “DATA_SYNC_NAME”), and then the value of the static field (i.e. “CRM_SYNC”).
142 )))
143 1. (((
144 Click “Add” to create additional static field appends.
145 )))
146
147 6.    Re-order Fields
148
149 1. (((
150 *Optional Step: If the order of the columns needs to be re-arranged for the order in which they appear on the export from a data feed: simply drag and drop the order of fields by clicking and holding the field labels.
151 )))
152
153
154 Transaction Mapping is done the same way as Demographic Mapping by using the drag and drop functionality to match Third-Party transaction values with ONEcount transaction values.
155
156 There are four Mapping Areas for Transaction Mapping:
157
158 * (((
159 Product Mapping Area
160 )))
161 * (((
162 Product Status Mapping Area (optional)
163 )))
164 * (((
165 Subscription Mapping Area
166 )))
167 * (((
168 Date Mapping Area
169 )))
170
171 **NOTE THAT MAPPING IS ID TO ID IN TRANSACTION MAPPING. THE ID IS FOUND IN PARENTHESIS AFTER THE TEXT IN THE "THIRDPARTY..." COLUMN OF MAPPING AREAS.**
172
173 **AS WITH DEMOGRPHIC MAPPING, ONECOUNT PRODUCTS, PRODUCT STATUSES, AND SUBSCRIPTION TYPES MUST BE CREATED PRIOR TO PERFORMING TRANSACTION MAPPING.**
174
175 **THIRD-PARTY TRANSACTION VALUES CAN BE PRE-POPULATED BY UPLOADING A LAYOUT FILE OR MANUALLY ENTERED (AS DESCRIBED IN "INFORMATION" SECTION ABOVE).**
176
177 *(% style="text-decoration:underline" %)Product Mapping Area(%%) *
178 **For automated data feed reports**, the Third Party Product column shows what the ONEcount product name will be displayed as on output.
179
180 * (((
181 For example, a user is subscribed to/registered for ABCXY Pharma Aug. Webinar(207) in ONEcount. The third party requires that on the report output show a value of August Webinar rather than ABCXY Pharma Aug. Webinar. Mapping ONEcount's ABCXY Pharma Aug. Webinar(207) product to August Webinar(400) will display the Third Party Product value of August Webinar on the output.
182
183 **Product Column Field:** This identifies what to output as the header for the product field. Enter the heading that's required for the report output.
184 )))
185
186 **For imports,** the Third Party Product column identifies the names/values from the import file that will be mapped to the ONEcount product.
187
188 * (((
189 For example, there's "product" column in the import file that identifies what product a particular record is subscribed to or registered for. In the import file, August Webinar in that colum indicates the record is subscribed to/registered for the August Webinar. In ONEcount, the product name is ABCXY Pharma Aug. Webinar. Any record in the import file that has August Webinar in the product column will be subscribed to the ABCYXY Pharma Aug. Webinar product in ONEcount.
190
191 **Product Column Field:** This identifies the column header for the product column in the import file. In the above example, you would enter "product".
192 )))
193
194 (% style="text-decoration:underline" %)**Product Status Mapping Area (Optional)**(%%)
195 **For automated data feed reports,** the Third Party Product Status column shows what the ONEcount product status will be displayed as on output.
196
197 * (((
198 For example, a user has a product status of "Not Active(405)" in ONEcount. The third party requires that the report output show a value of Inactive, rather than Not Active. Mapping ONEcont's "Not Active(405)" status to "Inactive(203)" will display the Third Party Product Status of Inactive on the output.
199
200 **Product Status Column Field:** This identifies what to output as the header for the product status field. Enter the heading that's required for the report output.
201 )))
202
203 **For Imports**, the Third Party Product Status column identifies what name/value from the import file will be mapped to the ONEcount product.
204
205 * (((
206 For example, there's a "product status" column in the import file that identifies the product status a particular record has for a particular product. In the import file, "Inactive(203)"in that column indicates the record has a product status of inactive. In ONEcount, the product status is "Not Active(405)". Any record in the import file that has a product status of "Inactive" in the product status column will have the product status of "Not Active" applied to the record in ONEcount.
207
208 **Product Status Column Field:** This identifies the column header for the product status column in the import file. In the above example, you would enter "product status".
209 )))
210
211 (% style="text-decoration:underline" %)**Subscription Mapping Area**(%%)
212 **For automated data feed reports,** the Third Party Subscription column shows what the ONEcount subscription type will be displayed as on output.
213
214 * (((
215 For example, a user has a subscription type of "n" in ONEcount. The third party requires that the report output show a value of new, rather than "n". Mapping ONEcont's "n" status to "new(new)" will display the Third Party Product Status of new on the output.
216
217 **Subscription Column Field:** This identifies what to output as the header for the subscription type field. Enter the heading that's required for the report output.
218 )))
219
220 **For Imports**, the Third Party Subscription column identifies what name/value from the import file will be mapped to the ONEcount product.
221
222 * (((
223 For example, there's a "Subcription Type" column in the import file that identifies the subscription type a particular record has for a particular product. In the import file, "New" in that column indicates the record has a subscription type of new. In ONEcount, the subscription type is "n". Any record in the import file that has a product status of "New" in the subscription type column will have the product status of "n" applied to the record in ONEcount.
224
225 **Subscription Column Field:** This identifies the column header for the subscription type column in the import file. In the above example, you would enter "Subscription Type".
226 )))
227
228 (% style="text-decoration:underline" %)**Date Mapping Area**(%%)
229 **For automated data feed reports,** enter the format you would like dates to be exported as. For example MM/dd/yyyy will export as 07/05/2023 and yy/MM/dd will export as 23/07/05. If you want to import time along with date add hh:mm:ss:a along with date. For example MM/dd/yyyy hh:mm:ss:a will be exported as 07/23/2023 11:45:32:AM
230
231 (% style="color:#333333" %)MM : months, dd: days, yyyy: years, hh: hours (12 hour format), HH: hours (24 hour format), mm: minutes, ss: seconds, a:AM/PM
232
233 **Date Column Field:** This identifies what to output as the header for the date field. Enter the heading that's required for the report output.
234
235 **For Imports,** enter the format of dates in the import file. For example, if a date is 07/05/2016, enter the format MM/dd/yyyy. If you want to import time along with date add hh:mm:ss along with date. For example MM/dd/yyyy hh:mm:ss:a will be exported as 07/23/2023 11:45:32:AM
236
237 (% style="color:#333333" %)MM : months, dd: days, yyyy: years, hh: hours (12 hour format), HH: hours (24 hour format), mm: minutes, ss: seconds, a:AM/PM
238
239 **Date Column Field:** This identifies the column header for the date column in the import file.
240
241 When done with mapping, click "Next".
242 You will then be brought either to Activity Mapping or Save, depending on selection made in 1 – Information.**SAVE MAPPING**
243
244 When done with mapping(s), click "Next".
245
246 You then have the option to save the mapping (Save) or return to mapping (Previous).
247
248 Once mapping is saved, you can find it on the Manage Mapping listing page.
249
250 A saved mapping is ready to be used for generating automated feed reports in the Dashboard. See Dashboard Reports – Create Feed Report for more information.
251
252
253
254 {{id name="End ScreenSteps Content"/}}
255
256 === MAPPING ENGAGEMENT DATA ===
257
258 This section outlines the types of engagement data that DExTr can import/export. Please note that Generic and Engagement data fields can be used for both web, banner and newsletter engagement data.
259
260 If you need data points imported or exported that are not on this list, check with the ONEcount team to see if those fields can be accommodated.
261
262 Generic Engagement Fields
263
264 * Activity Date: Date of the Activity
265 * Activity Time: Time of the Activity, Mandatory for Import.
266 * Channel Codes: Codes for Email, Banner and Web page views, CMI needed it in export.
267 * End Date: End date of export, mainly used in aggregate export.
268 * Unique External ID: Unique ID for each transaction.
269 * Month: Month of the Activity.
270 * OCID:
271 * OCID Hash: Unique hash of the user. This is the cookie set on user’s browser.
272 * Platform Viewed: Platform of the user in which Impression or Page view happened.
273 * Resource Details: This is used for export. It has Page title for Web activity, Banner name for Banner Activity and Message title for Email data.
274 * Start Date: Start date of export, mainly used in aggregate export.
275
276 Metric Engagement Fields
277
278 * Activity Metric code: Metric codes for Activity data. It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view. It is mandatory for Import and export.
279 * Activity Metric Name: Metric Names for Activity data (CMI report needs Metric code and Name). It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view.
280 * Activity Metric count: Mainly used for export. It has counts for each activity.
281 * Aggregate Metric code: Metric codes for Aggregate data. It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view. Mandatory for aggregate export.
282 * Aggregate Metric Name: Metric Names for Aggregate data (CMI report needs Metric code and Name). It has codes for Email sent, delivered, opened, clicked, banner impression, banner clicked and page view.
283 * Aggregate Metric count: Mainly used for export. It has counts for each activity.
284
285 Banner Engagement
286
287 * Ad Server Name: Name of the Ad Server, i.e., DFP
288 * Advertiser ID: ID of the Advertiser
289 * Advertiser Name: Name of the Advertiser
290 * Banner ID: ID of the Banner
291 * Banner Name: Name of the Banner
292 * Campaign ID: ID of the Campaign
293 * Campaign Name: Name of the Campaign
294 * Viewable Impression: If Impression is viewed or not, Boolean (0, 1 accepted)
295
296 Newsletter Engagement Fields
297
298 * Click URL: URL of the Clicked link. Mandatory for Importing Email clicks.
299 * List ID: ID of the newsletter List of Group
300 * List Name: Name of the newsletter list or group
301 * Message ID: Campaign ID of the newsletter
302 * Message Name: Campaign name of the newsletter
303 * Recipient ID: Unique User Id in the Email service provider
304 * Message Subject: Subject of the Campaign
305
306 Web Engagement Fields
307
308 * Referrer: Referrer for the page view
309 * Site name: Website URL
310 * Page URL: URL of the Page.
311
312 === **HOW DExTr MAPS DATA** ===
313
314 The following information is being provided to help you understand how DExTr will map/transform your data on import.
315
316 (% class="diff-block-target" %)
317 1. On import, ONEcount will always check if fields in the DExTr mapping are present in the file or not; if any field is not present in the file, then we fail the job. This is to ensure that we won’t miss mandatory data or dedupe fields and possibly corrupt the data. If you are importing data and the file does not have all of the fields in the DExTr mapping you are using, you can create a blank column with that field name in it–//as long as that field is not a de-dupe field for the import.//
318 1. If there are multiple dedupe fields in mapping then we dedupe the data based on order that the fields appear in the mapping. For Example if mapping has dedupe fields as “Email” (1^^st^^ field in order) and “AccountID” (2^^nd^^ field in order) then DExTr will first query the datase for a match on “Email,” if match is not found then it will query on “AccountID”. It always an OR match. You //can not match// on Email AND AccountID
319 1. If OCID is present as a dedupe field, then it always takes precedence irrespective of order. If you import a file with OCID as a dedupe field, and a record does not have an OCID, then a new user will be created in ONEcount and a unique OCID will be generated.
320 1. There are a variety of validations which can be added to DExTr mapping like, “Numeric”, “Required” etc.. If any record/row doesn’t satisfy these validations, then that record won’t be imported into ONEcount. It will be saved into a failed record file and mailed to the email address supplied with the import job.
321 1. If there are any passwords that are being imported, they should be imported as plain text. ONEcount will encrypt the field based on encrypt function for your installation.
322 1. For Package transactions: Term ID, Package status, source code and transaction type are mandatory.
323 11. If Term ID is not mapped, then the transaction won’t be imported.
324 11. IF Package status is not mapped then default Package status from term will be added.
325 11. If source code is not mapped. Then default source code from import job will be selected.
326 11. If transaction type is not added, then default type from Import job will be selected.
327 1. For Product transaction: Product ID, source code and transaction type are mandatory.
328 11. If Product ID is not mapped, then the transaction won’t be imported.
329 11. If source code is not mapped. Then default source code from import job will be selected.
330 11. If transaction type is not added, then default type from Import job will be selected.
331 1. If Transaction Date is not mapped or not formatted correctly, then current date will be used as request date.
332 1. If Expiration date is not mapped or not formatted correctly, then expiration date will be calculated based on the duration of the term and the request date.
333 1. For Engagement imports, Activity type is mandatory.
334 1. If Engagement date is not formatted, then current date will be used as activity date.
335 1. If Data and campaign ID, name, List ID and name then they will be imported as well.
336 1. If dedupe is not added/matched for an engagement import then new users will be created in the system just like Users and transaction import.
337
338
339 {{/layout-cell}}
340
341 {{layout-cell}}
342 {{panel title="More System UTILITIES"}}
343 {{expand title="Expand"}}
344
345
346 (% style="margin-left: 30.0px;" %)
347 [[Job List>>doc:Utilities - Job List]]
348
349 (% style="margin-left: 30.0px;" %)
350 [[Backend User Permissions>>doc:Utilities - Backend Users and Permissions]]
351
352 (% style="margin-left: 30.0px;" %)
353 [[Email Engine Sender Score>>doc:ONEmail and Sender Score]]
354
355 (% style="margin-left: 30.0px;" %)
356 [[doc:ONEmail]]
357
358 (% style="margin-left: 30.0px;" %)
359 Action Manager
360
361 (% style="margin-left: 30.0px;" %)
362 [[Configuration Manager>>doc:Utilities - Configuration Manager]]
363
364 (% style="margin-left: 30.0px;" %)
365 Backend Activity Log
366
367 (% style="margin-left: 30.0px;" %)
368 IVT Mitigator
369
370 (% style="margin-left: 30.0px;" %)
371 [[Widget Builder>>doc:Utilities - Widget Builder]]
372
373
374
375 (% style="margin-left: 30.0px;" %)
376 Import/Export Tool
377 {{/expand}}
378 {{/panel}}
379
380
381
382 {{/layout-cell}}
383 {{/layout-section}}
384 {{/layout}}