DeepMind’s deal with the NHS investigated

Android, Google, silicone valley

An academic paper has suggested that there were serious “inadequacies” in the deal between Google’s Artificial Intelligence company DeepMind and the NHS.

The idea was to build an app that would alert doctors to patients that were at risk of acute kidney injury (AKI), with over a million patient records being shared with DeepMind.

Google’s DeepMind said that the report contained “major errors”.

The company said they would be performing its own analysis, which was welcomed by the authors of the paper.

The deal between London’s Royal Free Hospital and DeepMind became public in February 2016, three months after the data collection began. Controversy was caused by the deal, as it raised concerns about the sharing of patient information without public consultation.

Mr Hodson, a former New Scientist journalist, and co-author Julia Powles, a Cambridge University academic, say there are still a great deal of questions to be answered.

“Why DeepMind, an artificial intelligence company wholly owned by data mining and advertising giant Google, was a good choice to build an app that functions primarily as a data-integrating user interface, has never been adequately explained by either DeepMind or Royal Free,” they wrote.

Currently, the app does not contain any AI, however DeepMind has plans to incorporate such
features in order to create smarter alerts.

Some of the criticisms of the deal include: questions as to DeepMind’s status as a data processor, given that it had created an app with a direct effect on patient care; an absence of legal documents regarding the use of data; and questions of correct registration.

In response, DeepMind and the Royal Free issued a joint statement: “This paper completely misrepresents the reality of how the NHS uses technology to process data.

“It makes a series of significant factual and analytical errors, assuming that this kind of data agreement is unprecedented.

“In fact, every trust in the country uses IT systems to help clinicians access current and historic information about patients, under the same legal and regulatory regime.”

The NHS has a number of information-sharing agreements with third-parties, however this is the first deal with a major US company.

Currently, an investigation by the Information Commissioner’s Office is taking pace, while it is also being looked into by the National Data Guardian, which is tasked with looking after health data.

In a statement, the ICO told the BBC: “Our investigation into the sharing of patient information between the Royal Free NHS Trust and Deep Mind is close to conclusion.

“We continue to work with the National Data Guardian and have been in regular contact with the Royal Free and Deep Mind who have provided information about the development of the Streams app.

“This has been subject to detailed review as part of our investigation. It’s the responsibility of businesses and organisations to comply with data protection law.”

The National Data Guardian added: “Our consideration of this matter has required a thorough approach in which the NDG and her panel have kept patients’ rightful expectations of both good care and confidentiality at the forefront of discussions.

“The NDG has provided a view on this matter to assist the ICO’s investigation and looks forward to this being concluded as soon as practicable.”

DeepMind has been desperate to stress that none of the data collected for the app has been shared with parent company Google.

AKI is linked to 40,000 deaths a year in the UK and can lead to many serious health concerns, ranging from the need for transplant at the more extreme end, to minor kidney dysfunction at the less serious end.

In February, DeepMind published details about how the app was improving patient care.

It revealed that more than 26 doctors and nurses at the Royal Free are now using the app, which is named ‘Streams’, and that each day it alerts them to 11 patients at risk of AKI.

Sarah Stanley, a consultant nurse who leads the patients at risk and resuscitation team, said: “Streams is saving us a substantial amount of time every day. The instant alerts about some of our most vulnerable patients mean we can get the right care to the right patients much more quickly.”