Data adaptors

A data adapter is any method, tool or process used to exchange data between two different systems. It can be very simple or very complex, but its function is to extract or read some data, and write an amended version after performing some transformation on it (sometimes known as ‘Extract, Transform and Load’). This might be:

• a calculation, for example expressing an absolute value as a percentage of the total, or changing date of birth to current age

• a conversion, for example converting inches to centimeters, date to datetime format, or easting-northing to longitude-latitude.

• a lookup, for example assigning towns to healthcare resourced catchment areas.

Box 1 illustrates how a data adapter can be used to standardise data.

Box 1: Using a data adapter to standardise date formats

Scenario: A dataset has been created from several sources that combine test results including the date the test was administered. However, the date format used by the datasets varies from row to row. If part of your project involves plotting these test results over time, this variation will make that difficult to do.

Start by choosing an appropriate standard date format and applying it to each value in the ‘test_date’ column.

Data as supplied

Data with the date in DD/MM/YYYY format

Now we have a dataset with a consistent date format, but that’s only half the job done. If we pass this data to someone else to work on, can they easily tell whether we’ve used DD/MM/YYYY or is it in fact MM/DD/YYYY? From the three rows shown here, it could be either. While this is consistent, it is not necessarily a good standard to use. A better choice is one of the ISO date formats, for example YYYY-MM-DD.

Data with the date in ISO 8601 format

Document an explanation of the data field and the standard applied, for example: test-date: Date of self-administered nasal swab; described in the format: datetime ISO 8601.