A new computer model links Google searches for terms like “stomach cramps” to smartphone location history data to determine which restaurants those people had recently visited, to identify potentially unsafe establishments for health inspections.
The method was significantly more accurate in identifying potentially unsafe restaurants when compared with existing methods of consumer complaints and routine inspections, according to new research led by Google and Harvard TH Chan School of Public Health. The findings indicate that the model can help identify lapses in food safety in near real time.
“Foodborne illnesses are common, costly, and land thousands of Americans in emergency rooms every year. This new technique, developed by Google, can help restaurants and local health departments find problems more quickly, before they become bigger public health problems,” said corresponding author Dr Ashish Jha, KT Li professor of global health at Harvard Chan School and director of the Harvard Global Health Institute.
Foodborne illnesses are a persistent problem in the US and current methods by restaurants and local health departments for determining an outbreak rely primarily on consumer complaints or routine inspections. These methods can be slow and cumbersome, often resulting in delayed responses and further spread of disease.
To counter these shortcomings, Google researchers developed a machine-learned model and worked with Harvard to test it in Chicago and Las Vegas. The model works by first classifying search queries that can indicate foodborne illness, such as “stomach cramps” or “diarrhoea.” The model then uses de-identified and aggregated location history data from the smartphones of people who have opted to save it, to determine which restaurants people searching those terms had recently visited.
Health departments in each city were then given a list of restaurants that were identified by the model as being potential sources of foodborne illness. The city would then dispatch health inspectors to these restaurants, though the health inspectors did not know whether their inspection was prompted by this new model or traditional methods. During the period of the study, health departments continued to follow their usual inspection procedures as well.
In Chicago, where the model was deployed between November 2016 and March 2017, the model prompted 71 inspections. The study found that the rate of unsafe restaurants among those detected by the model was 52.1% compared with 39.4% among inspections triggered by a complaint-based system. The researchers noted that Chicago has one of the most advanced monitoring programmes in the nation and already employs social media mining techniques, yet this new model proved more precise in identifying restaurants that had food safety violations.
In Las Vegas, the model was deployed between May and August 2016. Compared with routine inspections performed by the health department, it had a higher precision rate of identifying unsafe restaurants.
When the researchers compared the model with routine inspections by health departments in Las Vegas and Chicago, they found that the overall rate across both cities of unsafe restaurants detected by the model was 52.3%, whereas the overall rate of detection of unsafe restaurants via routine inspections across the two cities was 22.7%.
Interestingly, the study showed that in 38% of all cases identified by this model, the restaurant potentially causing foodborne illness was not the most recent one visited by the person who was searching keywords related to symptoms. The authors said this is important because previous research has shown that people tend to blame the last restaurant they visited and therefore may be likely to file a complaint about the wrong restaurant. Yet clinically, foodborne illnesses can take 48 hours or even longer to become symptomatic after someone has been exposed, the authors said.
The new model outperformed complaint-based inspections and routine inspections in terms of precision, scale, and latency (the time that passed between people becoming sick and the outbreak being identified). The researchers noted that the model would be best leveraged as a supplement to existing methods used by health departments and restaurants, allowing them to better prioritize inspections and perform internal food safety evaluations. More proactive and timely responses to incidents could mean better public health outcomes. Additionally, the model could prove valuable for small and mid-size restaurants that can’t afford safety operations personnel to apply advanced food safety monitoring and data analysis techniques.
“In this study, we have just scratched the surface of what is possible in the realm of machine-learned epidemiology. I like the analogy to the work of Dr John Snow, the father of modern epidemiology, who in 1854 had to go door to door in Central London, asking people where they took their water from to find the source of a cholera outbreak. Today, we can use online data to make epidemiological observations in near real-time, with the potential for significantly improving public health in a timely and cost-efficient manner,” said Evgeniy Gabrilovich, senior staff research scientist at Google and a co-author of the study.
Machine learning has become an increasingly powerful tool for solving complex problems, and its application in public health has been underutilized. The objective of this study is to test the efficacy of a machine-learned model of foodborne illness detection in a real-world setting. To this end, we built FINDER, a machine-learned model for real-time detection of foodborne illness using anonymous and aggregated web search and location data. We computed the fraction of people who visited a particular restaurant and later searched for terms indicative of food poisoning to identify potentially unsafe restaurants. We used this information to focus restaurant inspections in two cities and demonstrated that FINDER improves the accuracy of health inspections; restaurants identified by FINDER are 3.1 times as likely to be deemed unsafe during the inspection as restaurants identified by existing methods. Additionally, FINDER enables us to ascertain previously intractable epidemiological information, for example, in 38% of cases the restaurant potentially causing food poisoning was not the last one visited, which may explain the lower precision of complaint-based inspections. We found that FINDER is able to reliably identify restaurants that have an active lapse in food safety, allowing for implementation of corrective actions that would prevent the potential spread of foodborne illness.
Adam Sadilek, Stephanie Caty, Lauren DiPrete, Raed Mansour, Tom Schenk Jr, Mark Bergtholdt, Ashish Jha, Prem Ramaswami, Evgeniy Gabrilovich