Sponsored content brought to you by
One of the seemingly daunting tasks of moving your data from an older complex system, or collection of spreadsheets to a new LIMS is a step that should be considered seriously, but can be a smooth process if approached in a considered manner.
A new system with exciting functionality for your laboratory will only be fully effective if the right data, in the appropriate format, is added.
Allocate resource to this part of the project
Data migration should be approached as something that will need its own plan and resources. This could either be an internal allocation of informed staff, or external contracting.
I would advise a combination is required. Your data is best understood and structured by you, but in combination with providers and consultants who have experience of many integration projects, the whole process can be managed and considered effectively.
This step of the process should very much be a collaborative effort between laboratory and provider to ensure the right decisions and data translations are achieved.
Is all of the data required?
In an age when metrics and reporting can maximise the value and usefulness of your data, it may be tempting to just transfer everything you have. This is worth some consideration though, as more data may increase the amount of time and work required to prepare it, and could complicate the process.
For example: notes on a sample that were important at the time they were added, but when considered in a new system structure are actually rendered unnecessary by new functionality. It may have been noted that this sample was derived from “Sample 235”, but this could be represented in sample linkage and genealogy rather than a note.
An alternative to transferring all data that is often overlooked is the option to view older data in a view or lookup from the new system. Rather than actually load the data, is it possible and feasible to search older less structured data when necessary? It may even be possible to simply attach some files as pictures or spreadsheets so that data can be accessed if it is ever needed but not integrated. If the data is accessible in a legacy system, or view then it can always be added with an additional import, or manually as required.
Control the data quality
Structure of data may be different in the new system, but that structure is now the required format and has to be respected in any transfer and ongoing additions.
A very simple example is mandatory fields. If some required data does not exist in older data, it will need to be entered, or a default value set to fit the new systems requirements.
A more complex example may be: some information that relates to multiple studies. In spreadsheet records, this could be studies highlighted in a color to indicate they are linked in this way. In the new world, either a linking entity in the database should be set up, or perhaps a more simplified entry field on each study record to indicate they all share some common information.
Consider the data required for going live
If the new system coming on line is mostly integral for some new projects that don’t require every piece of old data, then prioritise:
- What data is needed?
- What data can be added as the work progresses?
- Can staggered uploads be considered to allow continuation of work on the initial priorities?
It is important to accept that all new data added to the new system will be in the format and structure required going forward. This means that a quicker start on appropriate work will start creating data which does not need any further integration. It will also give you a useful template for other data you are trying to manage for upload.
Try not to be too focused on the visibility of the data at this stage. i.e. who should/shouldn’t see particular records and fields. This is something to be taken care of in the functionality and security elements of the new system, rather than in the integration planning stages.
Overall once you have some data in the new system, and processes in place for your operations the return on time invested will become apparent. Efficiency, reporting and use of the data that may once have been time consuming should now be intuitive or potentially automated.
