Tag: Olin Brookings Commission



Members of the 2021-22 Olin Brookings Commission present policy recommendations to an audience at the Brookings Institution on April 27, 2022. Commission members from left: The Hon. Mary Bono, Dr. Ann Marie Dale, Van Ingram, Gina Papush, Darrell West and Anthony Sardella, commission chair.

Before an audience of policymakers, journalists, scientists and healthcare professionals at the Brookings Institution in Washington, DC—plus dozens more who joined virtually or from a St. Louis watch party—researchers and members of the Olin Brookings Commission outlined their solutions and recommendations to tackle a troubling aspect of the opioid epidemic.

The six-member commission convened multiple times over the past 12 months and on Wednesday presented its work—along with a 53-page overview of the research process, policy recommendations and context—during a midday event at Brookings. The presentation outlined AI-driven tools to curb misdirection of opioid shipments and policy recommendations design to facilitate the use of these tools.

“As I listen to the presentation, I have this sick feeling in my stomach, thinking if we had these tools 10 years ago, how many lives could we have saved?” said Van Ingram, one of the members of the Olin Brookings Commission and executive director of the Kentucky Office of Drug Control.

The group was the first convened by WashU Olin as part of a partnership with Brookings underwritten by The Bellwether Foundation. The project was designed to explore quality of life issues in communities and recommend policy changes to address them.

Focus on diversion of prescription drugs

The inaugural 2021-22 project tackled the opioid epidemic and, more specifically, the illicit diversion of prescription opioids that exploited blind spots in the distribution supply chain, fueling decades of dependency and death. Once researchers had zeroed in on a data-driven answer to that problem, the six-member commission devised a series of policy recommendations to facilitate their use.

“The blind spots still exist,” said Anthony Sardella, the chair of the commission and a member of the research team. “Our goal: Can you use data science to remove these blind spots? With that focus our research was begun.” (See a full list of the 2021-22 Olin Brookings Commission members on the commission’s website.)

According to some reports, more than 100 billion prescription hydrocodone and oxycodone pills were distributed in the United States between 2006 to 2014.

In 2020 alone, approximately 69,700 people died of overdoses involving opioids in the United States.

An AI-driven solution

Olin researchers from the school’s Center for Analytics and Business Insights focused on the patterns of diversion within the drug supply chain using advances in data collection, data mining, artificial intelligence and machine learning. The solution: Olin researchers developed a suite of anomaly detection tools to identify diversion trends in data submitted to a database maintained by the US Drug Enforcement Administration.

Using historical data from the Automation of Reports and Consolidated Orders System (ARCOS) database from 2006 to 2012, including more than 400 million opioid transactions and 277,000 buyers, researchers developed a tool to flag and stop fraudulent opioid shipments before they are diverted. The team identified patterns among likely diverters and tested their findings against a known database of convicted buyers.

The tool is designed to flag future diverters with 100% precision accuracy (i.e., if the tool flags a buyer as a diverter, it is almost guaranteed that the prediction is correct). In other words, the tool will not produce false positives. The team achieved that level of precision accuracy because the tool “lives with” a moderate degree (51%) of recall accuracy (i.e., the tool catches about one-in-two diverters). In other words, the team was willing to live with a higher rate of false negatives to ensure 100% precision accuracy in flagging likely diverters.

Values-based, data-driven work

“This work is emblematic of what WashU Olin Business School represents,” said Mark P. Taylor, the Olin dean who initiated the work to gain the Bellwether grant and partner with Brookings. “We’re dedicated to applying the rigorous use of data and the careful consideration of our principles to go beyond the bottom line, to address and impact critical issues in society.”

Once the research team locked down its anomaly detection tool, the 2021–22 Olin Brookings Commission developed a series of policy recommendations that, in combination, can overcome existing policy obstacles to empower industry and government to work together and implement the team’s near real-time detection and alert system to thwart opioid diversion in the supply chain.

The 14 recommendations include establishing a daily or near real-time pilot for integration of the anomaly detection tool to test the operational methods and modernizing the ARCOS technology infrastructure to support daily or near real-time data entry by registrants. Download the full report of the 2021-22 Olin Brookings Commission.

And the work is not done. The research team intends to further refine its model to potentially look for additional flags—and even techniques for flagging the movement of nonprescription opioids.

“We can determine whether a transaction is supposed to happen or not,” said Annie Shi, a member of the research team and a marketing PhD student at WashU. “For example, if DEA receives a new transaction request, then our model will be able to predict if that transaction is supposed to happen or not. If it is predicted to be suspicious, the DEA will be able to hold off that shipment until further actions are taken.”

Pictured above: Members of the 2021-22 Olin Brookings Commission present policy recommendations to an audience at the Brookings Institution on April 27, 2022. Commission members from left: The Hon. Mary Bono, Dr. Ann Marie Dale, Van Ingram, Gina Papush, Darrell West and Anthony Sardella, commission chair.




Top row, Seethu Seetharaman, Michael Wall, Anthony Sardella; bottom row, Annie L. Shi, Chenthuran Abeyakaran.

Data scientists from WashU Olin have developed a process for flagging suspicious transactions across 100 pharmaceuticals—a process with a stunningly high level of precision and one that can immediately take aim at curbing the country’s decades-long opioid epidemic.

Working with a US Drug Enforcement Administration database that tracked six years’ worth of pharmaceutical transactions, the five researchers developed an “anomaly detection” system that could flag future suspicious shipments with 100% precision.

In other words, as the researchers noted, when their process says a transaction is suspicious, it is. Basically, their anomaly detection system doesn’t flag a transaction unless it’s sure—which does mean some bad buys could sneak under the radar if they don’t meet the system’s criteria.

“The signals of anomaly detection are very strong for these egregiously suspicious buyers,” the study’s authors wrote. “This renders our algorithm very valuable for practical use.”

Built to guide the fight

The system was conceived as a tool to help deploy limited resources as authorities tackle illicit trafficking in narcotics.

“Having 100% precision is a very important feature of our (process),” the research team wrote in its paper, under review with the Journal of Marketing. “We are willing to sacrifice some recall (and increase false negative errors) in order to enable the practical adoption of our proposed algorithm.”

The research team—all associated with Olin’s Center for Analytics and Business Insights—includes Annie L. Shi, a doctoral student in marketing; Seethu Seetharaman, co-director of CABI and Olin’s W. Patrick McGinnis Professor of Marketing; Anthony Sardella, CABI senior research advisor; Michael Wall, co-director of CABI and a professor of practice in marketing; and Chenthuran Abeyakaran, BS ’21/SI ’23.

Their work comes under the auspices of the Olin Brookings Commission, a project operated by WashU Olin and the Brookings Institution to address critical policy issues affecting communities. The project is funded through a grant from The Bellwether Foundation.

Organizers of the first commission under the Bellwether grant focused on the opioid epidemic that’s killed half a million individuals in the US in the past two decades, according to the Centers for Disease Control. In July, the federal government reached a $26 billion settlement with the country’s three major drug distributors and pharmaceutical giant Johnson & Johnson for their roles in the epidemic.

“Addressing this issue and enabling distributors to have a predictive system that can be used to flag and halt suspicious orders of opioid drugs, is the central focus of this study,” the research team wrote in its paper, “Nip it in the Bud! Managing the Opioid Crisis: Supply Chain Response to Anomalous Buyer Behavior.”

Training the anomaly detector

The team “trained” its anomaly detection system by using a recently released DEA database called Automated Reports and Consolidated Ordering System—or ARCOS.

That database tracked millions of prescription drug transactions—their manufacture and distribution—spanning 2006 to 2012. By zeroing in on opioid transactions, with the guidance of a smaller database of known illicit transactions, the research team identified patterns of behavior across 40 different criteria. The scholars also developed a standard they called “morphine milligram equivalents”—or “MME”—to create reliable comparisons among various opioid transactions.

Ultimately, they found that seven criteria were enough to create an extraordinarily precise tool to flag suspicious transactions. For example, when looking at “average MME purchased per transaction,” suspicious buyers purchased almost 10 times as much as legitimate buyers. When they looked at “median MME purchased per transaction,” suspicious buyers purchased almost 20 times as much.

In the context of the research team’s detection and alert system, members of the Olin Brookings Commission will likely investigate proposals that affect public policy affecting the trafficking of illicit narcotics. Some of those policy areas could include:

  • data sharing and cross-agency communication;
  • revised and modernized data reporting;
  • funding sources and spending needs for system maintenance;
  • response guidance when transactions are flagged.

Pictured at top: top row, Seethu Seetharaman, Michael Wall, Anthony Sardella; bottom row, Annie L. Shi, Chenthuran Abeyakaran.




WashU Olin’s Center for Analytics and Business Insights is on the cusp of creating a machine-learning tool to flag suspicious opioid sales, just as government lawyers announced a multibillion-dollar settlement against three major drug distributors—a settlement that requires a database to track the destination of every opioid dose.

Analyzing a database of more than 400 million opioid transactions from the US Drug Enforcement Administration—a database that includes 277,000 buyers from 2006 to 2012—Olin researchers are building an algorithm that would help law enforcement officials identify shady opioid transactions in the future. The Olin scholars are working to understand key differences in the characteristics and behaviors of convicted buyers who they have identified in the data set to that of unconvicted buyers to inform their model-building approach.

“We want to ‘learn’ what variables distinguish the ‘bad’ buyers from the ‘good’ buyers,” said Seethu Seetharaman, Olin’s W. Patrick McGinnis Professor of Marketing and co-director of the Center for Analytics and Business Insights. “Once we learn the important variables that distinguish bad buyers from good buyers, we train a machine-learning algorithm to take these variables for a given buyer and give a probability score of that buyer being a bad buyer.”

Research to support policy recommendations

Seetharaman, along with CABI co-director Michael Wall, is collaborating with Luoyexin (Annie) Shi, an Olin PhD student in quantitative marketing, on the analysis. The research underpins the first of three projects by the Olin-Brookings Commission. This first project centers on the opioid crisis and what policy measures are needed to confront it long-term.

The entire initiative was made possible by a $750,000 grant from The Bellwether Foundation Inc. This first commission, like the next two, is charged with tackling topics affecting the quality of life for people in St. Louis and across the country.

Seetharaman said the team’s work on the DEA data has quickly shown promise as a law enforcement tool to flag transactions that divert often legitimate prescription therapies toward illicit uses.

“Using the predictive algorithm, the DEA could predict a buyer’s probability of being a bad buyer,” he said. “This way, the DEA can allocate their human and capital resources wisely among high-value leads.”

A well-timed approach

The results come just weeks after lawyers for states, cities and counties plagued by staggering numbers of opioid deaths announced a tentative $26 billion settlement against three distributors of pharmaceutical painkillers: McKesson, Cardinal Health and AmerisourceBergen. The settlement would also include Johnson & Johnson, which no longer supplies raw material for opioids or sells such painkillers in the United States.

“Under the deal, the three distributors, which control 85 to 90% of the market, are required to establish and fund a ‘clearinghouse’ that shows where every opioid dose is headed,” The Washington Post wrote in its report on the settlement. “They must check the database before sending out each shipment of pills and hold theirs back if it appears that the recipient is asking for an extraordinary amount of drugs, a typical sign that some are being diverted and sold on the street.”

According to The Post, more than 100 billion prescription hydrocodone and oxycodone pills were distributed in the United States from 2006 to 2014. Last year, approximately 69,700 people died of overdoses involving opioids in the United States.

Shi said the database she’s analyzing—known as ARCOS, or Automated Reports and Consolidated Ordering System—covers the sale of 14 main varieties of opioids. Those can be further broken down into 170 kinds of substances, and further broken down into 9,133 different products.

The inaugural Olin-Brookings Commission includes a dream team of data scientists, law enforcement authorities, medical professionals and addiction experts with years of industry and policy experience between them. Commission chair Anthony Sardella—founder of evolve24, Olin faculty member and CABI research advisor—serves as a critical conduit between research efforts and the expertise of the commission.

In their current project, supported by CABI, the group is charged with identifying strategies for combatting the epidemic of opioids and recommending any changes in local, state and federal policy that might help curb the problem and sharpen the response from experts.

The commission’s next meeting is set for August 19. The group intends to issues its final report and policy recommendations in early 2022.




Suppose the epidemic of opioids plaguing the United States could be stopped at the source? Suppose 21st century technologies such as data mining, artificial intelligence and machine learning could flag risky drug shipments before they land in the hands of at-risk populations?

How could it be done? And what changes in local, state and federal policy would be required to curb the problem and sharpen the response from experts in law enforcement, public health and industry?

These questions form the heart of a new initiative between WashU Olin Business School and the Brookings Institution. Broadly speaking, the Olin Brookings Commission is a three-year initiative designed to recruit a dream-team of policy experts and scholars each year who will deeply analyze a single major policy issue and issue policy recommendations.

Made possible by a $750,000 grant from The Bellwether Foundation Inc., each commission will be charged with tackling topics affecting the quality of life for people in St. Louis and across the country. Each year’s panel will issue practical and realistic recommendations informing business strategy and public policy.

“We are pleased to provide multiyear support for the Olin Brookings Commission,” said Ginger Smith, president of The Bellwether Foundation. “Funding an initiative that deepens the partnership between Olin and Brookings, two leaders in their industries, is where we believe we can make an impact.”

The focus of our first commission

Our first commission convenes this month. This first six-member commission—in partnership with Olin’s Center for Analytics and Business Insights—will demonstrate how new technologies can curb opioid trafficking and potentially more than 100 other equally destructive examples of illicit trafficking.

At the same time, the commission will evaluate existing policy obstacles and reveal opportunities where policy changes can enable industry and government to implement a real-time detection and alert system across industry and government agencies.

“The initiative is very compelling. It leverages new advances in artificial intelligence and machine learning to proactively detect suspicious opioid orders before they are shipped,” said Anthony Sardella, chair of the first commission and founder of data insight firm evolve24. “This effort holds the promise to save lives, enhance public health and protect our vulnerable populations.”

An initial phase of the opioid research project involves mining a relatively new database from the US Drug Enforcement Agency: the Automated Reports and Consolidated Ordering System. CABI co-directors Seethu Seetharaman and Michael Wall, along with Tony (who also serves as CABI’s senior research advisor), will lead the data analysis portion of the project.

“I am excited that CABI is involved in such a high-stakes national policy-related initiative in terms of showcasing the analytics talent resident in Washington University in St. Louis,” Seethu said. “This could not fit more perfectly with the values-based, data-driven mission of Olin.”

Another key component of the Olin Brookings Commission is involvement from students, who will serve as “commission fellows” in research and logistical support for each project. Olin PhD marketing student Annie Shi will collaborate with Tony, Michael and Seethu and together, they will be co-authors on all publications that arise from this initiative.

Meanwhile, I’m pleased to announce that our first commission includes heavy hitters from the pharmaceutical industry, academia, law enforcement and advocacy organizations focused on drug policy. Find the list of commission members at the bottom of this column.

A signature program?

The Bellwether grant makes possible a long-held vision of mine, an extraordinary opportunity to further leverage and expand Olin’s powerful relationship with Brookings, while also convening thought leaders who can provide guidance and direction on “megatrends” in global business and public policy.

We envision that each commission’s report—targeting the White House, regional and national government policymakers and the media—would coincide with the springtime Olin MBA capstone experience with Brookings. That is our timeline for a report on the opioid project.

Commission members will convene in a series of virtual meetings—at least while the pandemic continues raging—over the course of this year.

In addition to recommendations influencing business practice and public policy, the initiative is structured to provide insightful, well-researched contributions to industry about societal megatrends, inform and influence the direction of future research and increase students’ knowledge about the confluence of business and public policy.

I’m confident that the Olin Brookings Commission can become one of Olin’s signature programs, further cementing our commitment to improving life in St. Louis—and changing the world, for good.

Members of commission No. 1, opioid trafficking

  • Anthony Sardella, founder, evolve24; faculty member, WashU Olin Business School. Commission chair.
  • The Hon. Mary Bono, board member, Community Anti-Drug Coalitions of America, former US representative.
  • Dr. Ann Marie Dale, assistant professor of medicine and occupational therapy, Washington University School of Medicine
  • Van Ingram, executive director, Kentucky Office of Drug Control
  • Gina Papush, global chief data and analytics officer, Cigna.
  • Darrell M. West, vice president and director, Governance Studies; senior fellow, Center for Technology Innovation, Brookings