This program is the first of three used in the De-Duplication logic which examines and combines multiple customer records that have been created for the same person. Refer to the chapter Overview - Data De-Duplication for more information about required set up steps and an overview of the process itself.


This program reads through the Customer File and "nominates" records that look like they should be merged together. After reading through the file, the program produces a report of the records selected. The customer record that has been nominated to be the surviving customer is noted with an asterisk. This program, in effect, queues up the records for processing.

Generally, the record with the most recent activity is the record nominated to be the survivor. However, certain Customer Types can be given more priority than others by entering a setting in a field in in Edit Customer Type Codes. A certain setting in the same field can be used to exclude certain Customer Types from consideration altogether.

WARNING: It is highly recommended that customer types for billing companies such as direct bill, insurance, and wholesale tour operators be EXEMPT from the de-duplication process until the user is intimately familiar with it. Including billing company types in de-duplication may change the billed party on a rental agreement.

User tip: Running this program again at a later time without having processed the records (in Process Data De-Duplication) will cause any new records to be added to the file. To delete a file that was already built but never processed, ask Thermeon's Customer Support Dept. to initialize the file.

To access the Select Records for De-Duplication program, type SELDUP (RET) at any menu "OPTION:" field or the appropriate line number from the Data De-Duplication Menu. The screen will display as follows:


OPTION: __ SELECT RECORDS FOR DE-DUPLICATION


1 Start Cus # FIRST
2 End Cus # LAST
3 Cust Type WLK
4 Obsolete Cust Type
5 License State ALL
6 Check License DOB Y
7 Check Name Addr Y
8 Check Name DOB Y
9 Check Name Addr DOB Y





1SAVE 2 3ERASE 4 5BACKUP 6 7HELP 8PRINT


Enter data as follows:

1. START CUST #

Enter the Customer Number of the first customer of the range to be processed or press RET to default to the first customer record on file.

EXAMPLE Type 5000 (RET)

2. END CUST #

Enter the Customer Number of the last customer of the range to be processed or press RET to default to the last customer record on file.

EXAMPLE Type (RET)

3. CUST TYPE

Enter up to 3 Customer Types of those records that should be examined for de-duplication. This field defaults to the Customer Type assigned to a first time renter as defined in the Edit Main Control Record.

Selecting "ALL" Customer Types should only be done once certain sensitive Customer Types have been flagged as excluded from consideration in the de-duplication process. See the WARNING in the introduction above.

EXAMPLE Type WLK (RET)

4. OBSOLETE CUST TYPE

Enter the Customer Type that should be assigned to the customer records that will become obsolete if they are merged into another customer record via the Process De-Duplication program.

EXAMPLE: Type PUR (RET)

Note: Once the de-duplication program has run and customer types have been changed, customer records with the obsolete type can be purged from the customer file.

5. LICENSE STATE

Fields 5 - 9 can be used to modify the standard logic used to nominate records. Before entering anything in these fields, please read the section HOW RECORDS ARE NOMINATED located later on in this chapter.

To limit the de-dup logic to customer records that have a single license state, enter that code in this field.

EXAMPLE Type (RET)

6. CHECK LICENSE DOB

The next 4 fields apply only if a license state was entered in Field 5. These 4 fields are yes/no fields.

If ONLY the license number and the date of birth should be used to nominate duplicate records (name and address will not be compared), enter Y in this field and N in Fields 7 - 9.

Y = yes

N = no

EXAMPLE Type (RET)

7. CHECK NAME ADDR


Enter

EXAMPLE Type (RET)

8. CHECK NAME DOB

Enter

EXAMPLE Type (RET)

9. CHECK NAME ADDR DOB

Enter t

EXAMPLE Type (RET)


The screen will then display:


OPTION: __ SELECT RECORDS FOR DE-DUPLICATION


1 Start Cus # 5000
2 End Cus # LAST
3 Cust Type WLK
4 Obsolete Cust Type PUR
5 License State ALL
6 Check License DOB Y
7 Check Name Addr Y
8 Check Name DOB Y
9 Check Name Addr DOB Y



1BEGIN 2 3ERASE 4 5BACKUP 6 7HELP 8PRINT


PRESS "F1"AT "OPTION:" TO BEGIN PROCESSING.


HOW RECORDS ARE NOMINATED

If Fields 5 - 9 are not changed (that is, they are left with the default entries), the following logic is used to nominate records for merging:

  1. If two records have the same Drivers license number, Issuing State and Date of Birth they will be nominated for merging.
  2. If two records have the same Last Name, First Name, Address Line 2, City, State and Zip, they will be nominated for merging.
  3. If two records have the same Last Name, First Initial, and Date of Birth, they will be nominated for merging.
  4. If two records have the same Last Name, Date of Birth, Address Line 2, City, State and Zip, they will be nominated for merging.

The program passes through the Customer File four times -- once for each of the tests outlined above. Records flagged for merging in one of the earlier tests are skipped on subsequent passes.

When records are nominated for merging, the system will generally flag the customer record with the most recent Modification Date as the survivor. Several duplicate records can be merged into a single surviving record. For example, if there are 4 records for "WLK" type customer John Smithfield from Portland in the customer file, the one that was most recently used will be flagged as the survivor, and the others will be flagged to be merged. After the merge, the 3 records other than the survivor will be considered obsolete and their customer type will be changed to the customer type entered in field 4 of this program.

Priority Codes 1 - 98: An exception to the Modification Date logic for determining the surviving record is made if Priority Codes have been set up for Customer Types. The Priority Code field in the Customer Type can be used to establish a hierarchy of Customer Types so that the one with the highest priority will be the surviving record. Priority Codes for the de-duplication process can range from 1 to 98 with 1 being the highest priority and 98 being the lowest. When these are in place, the Customer Type with the highest priority code takes precedence for the survivor, and the modification date is ignored during the process. If the records that are nominated for merging are the same level priority, then the modification date logic is used to determine which one will survive.

Priority Code 99: Customers who have a Customer Type that has a "99" as their priority are EXCLUDED from the De-Duplication process.

User tip: The field to validate customer types in Edit Misc. Control File (page 2, Field 7) must be set to "Y" in order for the priority logic to work.


DE-DUPLICATION REPORT

Once the nomination process is done, a report is automatically produced listing all records nominated for merging. The report can also be run (printed or displayed) at any time by pressing F8 from this screen or from the Edit Records for De-Duplication program.

All duplicate records for the same customer are listed as once group with the surviving record first. All others nominated to be merged into the record are listed next. A blank line separates the records for different customers. The data printed is the same data the program used to select duplicate records. That is:

Last Name
First Name
Drivers License #
Drivers License Issuing State
Date of Birth
Address Line 2
City
State
Zip

Examine this report !!

Use the Edit De-Duplication Records program to edit this list, delete records from the list, or manually enter other records.