Chasing a $10,000 award, Perelman School of Medicine Assistant Professor of Medicine Jaya Aysola, MD, MPH; med student Nancy Haff; behavioral economics research coordinator Cary Hilbert, and Perry Wilson, MD, MSCE, present their idea at a University of Pennsylvania innovation tournament in Cape May.
Should academic researchers use innovation tournament strategy to improve and speed the development of their pilot study proposals? That question was asked and answered at a recent Cape May, N.J., gathering of 50 health policy researchers from the University of Pennsylvania and Carnegie Mellon University.
In fact, after only two and a half hours of brainstorming and idea pitching, two teams each walked away with a $10,000 grant to fund their proposed study ideas for a novel use of credit card incentives and creative enhancement of health behavior study participants' enrollment options.
"This was our first experience taking innovation tournament theory into the research realm to do idea generation," said Kevin Volpp, Perelman School of Medicine Professor of Medicine and Director of Penn's LDI Center for Health Incentives and Behavioral Economics (CHIBE). "It worked out really well with a lot of energy and team building that generated many ideas that were quite good."
Catherine Maclean, PhD, Assistant Professor in the Perelman School of Medicine's Department of Medical Ethics and Health Policy; Jingsan Zhu, MS, MBA, CHIBE's Assistant Director of Data and Analysis; and Kevin Volpp, MD, PhD, discuss their pilot project idea.
Last year, CHIBE helped to organize a large-scale innovation tournament that solicited ideas to improve operations within Penn's $4.3 billion hospital and clinic network. More than 1,700 ideas were submitted by Health System staffers and those of two winning teams were funded for development. The basic concept used in Penn's innovation tournaments is driven by the work and book of Wharton School professors Christian Terwiesch and Karl Ulrich.
Cape May retreat This month, at its annual two-day Penn-CMU Roybal Center for Behavioral Economics and Health retreat at Cape May's Congress Hall, CHIBE conducted an experiment exploring how innovation tournament methodology might be applied to the scholarly research process. The focus was on finding a better solution to one of the biggest problems those researchers face when setting up new behavioral studies: recruiting a sufficient number of participants.
Such behavioral studies analyze issues like why diabetes patients are unable to maintain daily practices that would better control their disease, or why elderly patients fail to take the prescribed medications that can dramatically improve their conditions, or what incentives might induce obese individuals to achieve sustainable change in their lifestyles and eating habits.
The Perelman School of Medicine's Scott Halpern, MD, PhD, MBE, pitches his team's time-limited incentives idea. The team was later merged into a larger finalist team that produced one of the tournament's winning proposals.
"The fundamental challenge with most research," said Volpp, "is that when you have an opt-in process enrollment rates will always be far below 100%. And those rates vary by a number of factors including the ease of enrollment, the ease of being a participant in a project, the degree of complexity, and the type of incentives involved. Higher rates of participation speed the efficiency of conducting a study and, from a scientific standpoint, are very important because higher participation better reflects the characteristics of the population we're trying to engage."
Increase recruitment The Cape May retreat innovation tournament was aimed at identifying new ideas to study as potential methods for increasing peoples' willingness to take part in these larger behavior-related studies.
To get the tournament rolling, 51 health services researchers were divided up into 17 teams of three. They were told that the winning team would get a $10,000 grant to fund a study of their idea's feasibility.
Statistics Professor Dylan Small, PhD; Hannah McLane, MD; clinical research coordinator Jaclyn Sadicario; and clinical research coordinator Chantell Ketchem hurry to finish their idea schematics.
Clustering around the perimeter of Congress Hall's ballroom and armed with large easel pads and colored Sharpie markers, team members vigorously championed their concepts. The verbal din, scribbling and drawing was furious. Within twenty minutes, the walls of the ballroom were plastered chest-high with a pastiche of emerging ideas.
Refining pitch ideas Each of the 17 teams then gave a brief presentation on the single idea that came out of their deliberations. Tournament leaders then combined teams that had similar ideas to produce half a dozen final teams with six to nine members who huddled back against the walls to further refine their ideas into a specific proposal.
In the end, a spokesperson for each of the final teams gave a brief explanation of their concept, after which all retreat attendees voted on which they thought were best.
Instead of one winner, two teams were tied and CHIBE announced that each would get $10,000 to pursue further investigation of their hypotheses.
David Asch, MD, MBA, Perelman School of Medicine Professor of Medicine and Executive Director of the Penn Medicine Center for Health Care Innovation (left) and Michael Detsky, MD, a Health Policy Research post-doctoral student were part of one of the tournament's two winning teams.
The first was titled "Time Limited VISA Card." As a recruitment incentive, it proposed offering potential behavioral study participants a prepaid VISA card with a set expiration date. The expectation was that this could be used in a variety of ways that might create a greater sense of urgency for participation.
Winning concepts The VISA Card team consisted of David Asch, MD, MBA, Executive Director of Penn Medicine Center for Health Care Innovation; Scott Halpern, MD, PhD, MBE, CHIBE Deputy Director; Michael Detsky, MD, Health Policy Research post-doctoral student; Ashley Kraybill, Research Coordinator, Penn's Alzheimers Disease Center; Yiwei Zhang, Wharton Applied Economics PhD student; Tori Ulrich, CHIBE Clinical Research Coordinator; and Hencheng Dai, Wharton PhD student, Operations and Information Management.
The second winning concept was entitled "Expanding The Choice Set: Changing Norms For Participation."
Team member Amelia Haviland, PhD, a Carnegie Mellon Professor of Statistics and Public Policy, explained that currently, potential behavioral study participants are offered only a binary choice: yes, they will participate or no, they won't. The team's idea is that if the number of potential choices was increased, it might affect the prospective participants' perceptions and some percentage of their ultimate decisions.
Katie Auriemma, ScB and a medical student at Penn's Perelman School of Medicine (left) and Amelia Haviland, PhD, a Carnegie Mellon Professor of Statistics and Public Policy, hone their team's pitch.
"There is a tendency for individuals to expect that when there are two options, about half of everybody will choose each," said Haviland. "This sets up a norm or expectation that 'what I do doesn't really matter.' But if there are four options and three of them are some kind of participation, people might tend to think a little bit more like 'Oh I bet a bunch of people do the other ones, too."
This, she said, could move people to a "middle" decision.
A 'pull toward the middle' "The idea," she said, "is that people experience a general pull toward the middle. If they're looking at three sizes of, say, coffee, many people don't want the smallest or the biggest one. They find comfort in the middle. So there is the possibility that more people will select something because it's the middle instead of being only one of two options."
"What we'd like to do is study this in two phases," she said. "The first would be a lab study trying to understand what version of this might work best. Then there's the question of exactly how many options do we want? Three? Four? Five? And what kinds of additional options would be most useful? That's the first stage. The second phase would be to bring what we learn from that lab work into real behavioral trials that are about to begin their enrollment."
The team consisting of behavioral economics project managers Dana Gatto, Amanda Schultheis, and Elizabeth Cooney; Katherine Courtright, MD; and Roy Rosin, MBA and Penn Medicine Chief Innovation Officer, present their social media-related recruitment idea.
Along with Haviland, the second winning team consisted of Andrea Troxel, ScD, Penn Center for Clinical Epidemiology and Biostatistics; Marilyn Schapira, MD, MPH, Penn Medicine Abramson Cancer Center; Eric VanEpps, CMU PhD student in Behavioral Decision Research; Shereen Chaudhry, CMU PhD student in Behavioral Decision Research;
Matt Eisenberg, CMU PhD student in Applied Economics and Management; Hilary Wolfendale, CMU PhD Student; Lisa Wesby, CHIBE Manager; Nancy Kennedy-Smith, Manager, Administration and Finance, Penn's Perelman School of Medicine; and Bradford Tuckfield, Wharton PhD student, Operations and Information Managment.
So, what was Volpp's end assessment of the tournament and its potential future place in scholarly research? Was it worth the $20,000 in grant prizes?
"I think this was really worth it because pretty much everybody at this retreat works on doing research all day and, on some level, is always thinking about ways we can improve recruitment," he said. "So the fact that in two and a half hours this tournament generated a number of exciting new ideas suggests that this process was actually quite efficient."
A measure of the promise Volpp sees in applying innovation tournament methods to scholarship is evident in the fact that CHIBE is organizing another tournament at the two-day "Applying Behavioral Economics to Perplexing Problems in Health and Health Care" conference it is co-hosting with the Robert Wood Johnson Foundation and the Donaghue Foundation on November 21 and 22 at the University of Pennsylvania.
~ ~ ~
Hoag Levins is a journalist and managing editor of LDI digitial publications.
