How Generic Data import works?

Under the administration menu, there is a Data import option that is accessible to users with an admin role. The idea behind Data import is that if the user has bulk data for any data asset ( Glossary, Data dictionary, Analytics, Fields & Lineage), then the user can load the data in the dvsum application using Excel files. Users can import up to 50,000 records through a .xls, .xlxs, or .csv file. Let's see how Data import work.

Watch this quick video tutorial on how Generic Data import works in application.

However, you can also follow these instructions as well:

Important: Ensure that the Domain and Sub-domain are created in the application before starting the import process. The import will use these as a reference when uploading data.

Step 1: 

Select the Data import menu option under the Administration tab, Now click on any data asset name:

Step 2:

To begin, download the sample file and fill in the data as per the instructions.

Important:

• For Field Dictionary imports, the Entity field should now include both the Data Domain (DD) and Subdomain (SD) in the format:
  DD.SD.EntityName
  For example: Default.Default.US_PASSPORT.
  This change ensures that the entity is uniquely identified during the import.

More information on above, refer to this article.

Additional Notes for Sample File:

• The Tags column in the sample file allows multiple tags to be added using the format {Tagset Name}={Tag}. Multiple tags can be separated by a comma. For example:Data Classification=Public, Custom Tags=PKR.
• The Data Steward and Data Owner fields in the sample file must contain valid email addresses of users already added in the DvSum tool.
• For Glossary Terms file imports only

  The Term Type column can have one of 8 predefined term types. The applicability of Business Rules (Category Data Rule, Pattern Rule, Regex Pattern Rule, Value Range Rule, and Manual) depends on the selected Term Type:

  • If the Term Type is Numerical, only Value Range Rule and Manual are applicable. For Value Range Rule, use the format {Min Value} - {Max Value}. For example (1-20)
  • If the Term Type is Categorical Attribute, the Category Data Rule should follow this format:
    {Categories.Value 1}={Categories.Description}, {Categories.Value 2}={Categories.Description}.
  • If the Term Type is Computed Measure, the Term Formula needs to be provided in the same file.

• Upload the sample file to import the data into the application. By default, the "Update Existing Records" button will be enabled for the Data Dictionary.

Upload the sample file to import the data in the application, by default Update Existing Records button would be enabled for the Data dictionary:

NOTE: User can Add New Records for Business glossary and Lineage only.

Users can Find existing records on the basis of Table ID or Source Name, Table. Then click on Next Button. 

NOTE: By Default "Don't update empty values" would be checked, so if uploaded file contain empty values,  it will not get updated in the dvsum application

Step 2.1:

Users can also download data from the listing page and import the downloaded file using Data import:

Step 3:

Fields from the uploaded file will get mapped with Field in dvsum. Mapped Tab will show all fields which are mapped. Once All Required Fields are mapped. Users can Click on Import button and start import

If a field is unmapped, the user can map it individually by selecting the appropriate option from the dropdown available in the Fields section under the DvSum column. Similarly, to remove a single field, the user can click the remove icon provided for that field.

For bulk actions, the user can utilize the Reset Field Mapping and Apply Auto Mapping buttons.

Step 4:

A success screen will be shown to the user when the file is imported successfully.

• I'm Done Button will redirect the user to the Data import step1.
• Go to Dictionary button will redirect the user to the respective dictionary listing page. 

Step 5:

After 3-5 minutes the user will receive an email with a complete summary of imported Data. Users can also download the summary of uploaded data using the Download button.