Key highlights and recommendations
This guidebook by the ODI is intended to inform the development of Smart Data schemes but does not represent formal government policy
The research process to create this guidebook involved extensive desk research, expert interviews across government, regulators, large corporates and third sector, and industry roundtables of existing and potential smart data service providers.
The key highlights and recommendations from this research are organised according to the ODI definition of Data Infrastructure - data assets, standards, technologies, policies and the organisations that steward and contribute to them. For more information on data infrastructure and its constituent parts, explore the next section of the guidebook.
Although data is central to developing solutions, innovations programmes must be problem-led, and not merely led by the data and technology available by key actors. Data assets in a Smart Data innovation programme should be scoped after a problem is thoroughly analysed.
Data innovation should be led by use cases that are co-designed with the end customer or those affected by the problem. This helps to build trust, and ensures that consumer needs are met, which ultimately defines the success or failure of any initiative.
Access to data is critical to modern, digital innovation. Both existing literature and user research suggests this component is of the highest importance to organisations seeking to build innovative new products, services and insights.
Using anonymisation techniques can allow access to data for Smart Data innovation to be provided through open publishing or secure sharing with trusted third parties
Data assets should be curated through consultation with data providers, data users and subject matter experts. For example: A climate action data innovation programme could focus on energy sector companies that are opening or providing access to Smart Data. The use cases could be scoped with input from those companies, potential Smart Data innovators, Ofgem (the regulator), climate scientists, renewable energy trade associations and consumer representatives.
Smart Data innovation programmes can help make data assets more valuable, by making them more trustworthy (e.g. through certification), relevant (e.g. through user-led design), and sustainable (e.g. through research into high value datasets).
Data innovation focussed on a few high detail datasets is often more valuable than that with many low detail datasets.
Standard development is paramount to Smart Data innovation, and should be done collaboratively across schemes, involving a diverse range of interests, and always have the end-users in mind.
Interoperability and standards are needed to build more open and efficient markets that allow new entrants to create innovative services, and consumers greater ease to switch service providers.
Most data innovation programmes do not necessarily build standards themselves, but Smart Data schemes should have standard development as a high priority. The Open Banking Standard has been valuable to providing the provision of new services in the finance sector, and the Open Energy Standards are leading the way in energy.
Smart Data innovation programmes can help to develop standards by understanding what works for real users, and help to test technical standards like data models, taxonomies, and technologies like APIs.
Data innovation programmes can be part of the standards development process. An objective of these programmes can be for data users and other stakeholders to test potential APIs, data models and other data infrastructure, to understand the of this infrastructure value to users and where it might need improving.
Many data innovation programmes deploy new technologies to help participants build their propositions, with generally mixed results.
Data access technologies like APIs are powerful tools to help data users innovate, but ensuring they are fit for purpose is an important part of innovation enablement.
Privacy enhancing technologies can be enablers for innovation, unlocking new opportunities for valuable data sharing and processing. Additionally, PETs can enhance privacy in existing projects, by enabling data sharing and processing to be carried out in a more privacy-focused way. The PETs Adoption Guide by CDEI can help Smart Data organisations on their journey.
The potential benefits of synthetic data are still not being fully realised, even in successful data innovation programmes such as the FCA Digital Sandbox and Nationwide’s Open Banking for Good.
Open source technologies have been increasingly seen as enabler technologies, both in Smart Data innovation programmes and in the wider market. Open APIs and OAuth allowed for better innovation sooner in Open Banking, and using open source technology allows for easier integration outside the innovation programme itself.
The term ‘sandbox’ is used loosely in different innovation contexts, but actual code testing environments as components of innovation models are seen as very useful.
The FCA Digital Sandbox Pilot, ICO Regulatory Sandbox and the Open Banking Directory Sandbox have been cited as examples of regulator-provided sandboxes that provided useful experiences to their partners, which they would not have been able to access otherwise. The success of the FCA pilot has led to an extension of the programme both in scope and duration.
The FCA is currently exploring that introduction of a Scalebox, including the creation of a ‘regulatory nursery’, in order to help scale-ups with growth and securing investment.
The GDPR right to data portability - which allows individuals to obtain and reuse their personal data for their own purposes across different services - is a powerful policy foundation for Smart Data. However without interventions like Smart Data schemes to build additional data infrastructure around it, it is seen as insufficient to deliver the intended impact.
Government has committed to legislation that places requirements on data holders to share data through Smart Data schemes and that gives the ability for regulators to enforce this policy. Although it is possible to use softer methods to achieve buy-in, these will take longer and may lead to uneven coverage within and across sectors. Sectors should have the legislative tools necessary and where they are needed.
Policies and guidance around the concept of ‘presumed open’ - that data should be published with an open licence unless it is sensitive or personal data - are improving access to data in the energy sector and laying the potential groundwork for a lot of potential innovation, as has been already shown with innovation programmes like Modernising Energy Data Access and Modernising Energy Data Applications. Exporting this concept to other regulated sectors could unlock growth in cross sector innovation.
Regulators providing support like the ICO’s ‘comfort from enforcement’ - which states that any inadvertent breaches of the UK GDPR during product or service development will not lead to immediate regulatory action on the basis that a collaborative dialogue is maintained with the Sandbox team - help enable data innovation by providing organisations with a safe environment to experiment, and regulatory steers to avoid legal breaches.
Regulators providing, but not mandating, technical guidance can help boost data innovation by enabling a smoother experience for innovators and end users.
Previous Smart Data schemes and initiatives have not foregrounded end-user needs and experiences enough, and more customer and community involvement in design is needed. Relevant communities should be involved in the innovation programme and model design process, including data providers, data users and end users or beneficiaries.
If a data innovation programme has particular groups as a target beneficiary, such as the Smart Data objective to help people in vulnerable circumstances, then those people and organisations representing them, such as debt charities and specialist health care organisations, should be involved in all stages of the programme. Involve end users to ensure their needs are met, whilst ensuring that there are protections and efforts in place to reduce risks.
Data innovation programmes that bring in a diverse mix of advisors, participants, and other key stakeholders create more attractive and valuable propositions. Partnering across public, private, and third sectors can build trust and interest in the programme. Smart Data schemes and other innovation programmes should explicitly include civil society organisations, charities and academia to cover gaps that for-profit tech companies will miss.
Programmes funding regulators have shown successes, but a lack of consistent funding to smaller regulators could be holding back cross-sector innovation.
Innovation programmes may need to spin up new organisations or data institutions to ensure the benefits of innovation remain sustainable, trustworthy and properly governed, such as with the Open Banking Implementation Entity.
Innovation programmes also have a role to play in upskilling the communities of interest in digital and data literacy, or sector specific information, such as around financial inclusion done by credit unions. Engaging and educating end users could help reduce the reputational risks for data providers and data users.
Last updated