The Data Migration Manager (DMM) provides a mechanism to bulk import and edit historical data, such as data loaded from Excel.
The DMM offers bulk validation and import of Locations, Wells, Field Chemistry, Historical Chemistry, Bore Logging and Geology data, as shown below. It also facilitates data transfer from other ESdat systems using the ESdat Data Interchange Format.
Data can be entered directly in the online worksheet, or blank Excel Import Templates can be downloaded, populated offline, then used to populate the worksheet in ESdat.
If you generate an Excel Import Template, once populated it will look like below. Columns with an asterix (*) in the header are required fields and must be populated with data. Data that is identified in Excel for importing will be visible by alternating blue and white rows as shown below.
Populated Excel Import Templates can be loaded, or data can be diretly pasted into the online worksheet.
All ESdat systems, including legacy ESdat Desktop systems, are able create an ESdat Data Interchange File which can be used to move data between ESdat systems.
A Data Interchange File can be loaded using the Data Interchange button, shown below.
The data will load in a series of sheets, ready for Validation and Import.
Drop-Downs are available to validate the data.
To import data the user will need to specify a Project for which they have Add Permissions. If the user is looking to update/overwrite existing data they will need to specifiy a project for which they have Edit permission.
Add | Add new records. Rows will only be processed if the record does not already exist. | |
Overwrite | Overwrite existing records. Rows will only be processed if the record already exists. Any existing data for the row will be overwritten, including with null values (meaning a value in the database will be overridden, potentially by an empty value). | |
Update | Update existing records. Rows will only be processed if the record already exists. Existing data for the row will be updated where a cell has a value. Empty cells will be ignored (meaning any existing data in the corresponding field already in ESdat will remain unchanged). | |
Add/Update | Add/Update existing records. Rows will be processed as an Update if the record already exists, otherwise they will be Added. |
Prior to importing the data into ESdat, you may choose to validate the data in your worksheet.
The validation checks for the following:
If any of these checks fail a summary the import errors are displayed to the user. The user can filter for specific errors, and a tooltip provides feedback on the error in each row as in the example below:
Successfully validated data will be imported, while rows with errors will be flagged and not imported. Once you have resolved any Validation issues, click the Import button to import the corrected data into ESdat.
If you are loading Chemistry data from Excel or an ESdat Data Interchange File you will need to ensure your data is using the standard ESdat ChemCode Reference List. For more information on loading Chemistry see the specific "Importing Historical Chemistry" document.
Every data import is automatically saved. It is also possible to add metadata to describe an import.
All saved imports and can be reopened. When it is no longer needed a saved import can be deleted.