Introduction > Data Tools > Processing

MRR File Processing

Email Comments

 

Simplified flow diagram for processing of MRR files.

 

Validate file format

Submitted MRR files are checked against the appropriate MRR file specification.

Reject file

If any validation checks fail, the entire file is rejected and no records are loaded into the database.

Validate operation

MRR files must be submitted with an operation of either Load or Correct to indicate how to process the file.

Load = Append records from this file to the database

Correct = Replace all records from the previously loaded version of this file with the records in this version of the file

If a file is submitted with an operation = Load, but a file with that name has already been loaded, it is an invalid operation.

If a file is submitted with an operation = Correct, but a file with that name has never been loaded, it is an invalid operation.

Validate submitter

The submitter's email is checked against a list of authorized submitters for that project code. If it is not on the list, then the submitter is invalid.

Check existing records and assign event types

Each record in the file is checked against records already in the database to identify duplicate records, orphaned records, and disowned records. See the MRR Data section for more information about event types.

Validate data values

Data values of each record are checked against the MRR data specification. Records which do not meet domain requirements are assigned a validation error or warning. Validation errors cause the file to be rejected, while validation warnings cause the file to be loaded with Provisional status.

Load records

Append new records or overwrite existing records as indicated by the Operation.

Send notification emails

Automated emails are sent to the MRR project coordinator and the submitter indicating the final status of file processing:

Loaded = all records were loaded; no validation warnings or duplicate records exist for this file

Provisional = all records were loaded, but some validation warnings or duplicate records exist

Rejected = no records were loaded due to file format, operation or data specification validation failures

Copy file to archive

The file is copied into the appropriate file archive directory and the file name is edited to record the load version of the file.

Update reporting system

Newly appended or updated records are pushed to the reporting system and available to query within 15-30 minutes of being received for processing.


Version: 1.07

Published: 9/6/2023