COVID-19 is an emerging, rapidly evolving situation.
Get the latest public health information from CDC:

Get the latest research information from NIH: Menu

A Pragmatic Trial of Two Strategies for Implementing an Effective eHealth HIV Prevention Program (Keep It Up! 3.0)

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been evaluated by the U.S. Federal Government. Know the risks and potential benefits of clinical studies and talk to your health care provider before participating. Read our disclaimer for details. Identifier: NCT03896776
Recruitment Status : Recruiting
First Posted : April 1, 2019
Last Update Posted : July 29, 2020
National Institute of Mental Health (NIMH)
Boston Medical Center
Weill Medical College of Cornell University
Marquette University
Information provided by (Responsible Party):
Brian Mustanski, Northwestern University

Tracking Information
First Submitted Date  ICMJE March 28, 2019
First Posted Date  ICMJE April 1, 2019
Last Update Posted Date July 29, 2020
Actual Study Start Date  ICMJE October 9, 2019
Estimated Primary Completion Date May 2023   (Final data collection date for primary outcome measure)
Current Primary Outcome Measures  ICMJE
 (submitted: March 28, 2019)
Public Health Impact (reach X effectiveness) [ Time Frame: 12 Months ]
Public Health Impact (PHI) involves the reach into the county's YMSM community, weighted by HIV risk (i.e., engagement of YMSM from higher risk communities is valued more), and effectiveness at reducing HIV risk, defined as condomless anal sex, STIs, and adherent PrEP use (i.e., three major risk factors for HIV expected to be modified by KIU!).
Original Primary Outcome Measures  ICMJE Same as current
Change History
Current Secondary Outcome Measures  ICMJE Not Provided
Original Secondary Outcome Measures  ICMJE Not Provided
Current Other Pre-specified Outcome Measures Not Provided
Original Other Pre-specified Outcome Measures Not Provided
Descriptive Information
Brief Title  ICMJE A Pragmatic Trial of Two Strategies for Implementing an Effective eHealth HIV Prevention Program (Keep It Up! 3.0)
Official Title  ICMJE A Pragmatic Trial of Two Strategies for Implementing an Effective eHealth HIV Prevention Program (Keep It Up! 3.0)
Brief Summary This study evaluates and compares two implementation strategies of an online HIV prevention intervention: Strategy 1 in which community-based organizations apply, and are selected, for funding to deliver Keep It Up! through current HIV testing programs; and Strategy 2 which is a "direct-to-consumer" model where centralized staff at Northwestern University recruit participants nationally through online advertising campaigns and manage engagement.
Detailed Description

Substantial effort has gone into developing and testing the efficacy of eHealth HIV prevention interventions, but these interventions will not affect the spread of the epidemic unless implemented effectively to reach the most at-risk populations. Despite emerging evidence that eHealth approaches can be highly effective in reducing HIV risk or improving care engagement, implementation requirements do not fit within established strategies for public health scale up, leaving it an open question as to how to bring these programs into practice. To date, little to no implementation science has examined strategies to effectively scale up eHealth HIV prevention programs.

The overarching goal of this study is to address this need by developing and evaluating novel strategies for implementing eHealth HIV prevention programs. Keep It Up! (KIU!) is an online program proven to reduce HIV risk among diverse young men who have sex with men (YMSM). KIU! was designed for YMSM who just tested HIV negative to keep it up and maintain negative status by reducing risk and enacting protections. Strong evidence of effectiveness on behavioral (decreased condomless sex) and biomedical outcomes (reduced STI incidence) makes KIU! an ideal eHealth intervention for implementation research. Lessons learned from KIU! implementation will set the pathway for implementing the many HIV eHealth programs currently undergoing efficacy testing.

Implementation strategies need to be evaluated using rigorous research designs. Through formative research with community-based organizations (CBOs), health departments, the CDC, and researchers, the investigators have identified two pragmatic implementation strategies that will be evaluated and compared:

In Strategy 1, CBOs apply for, and are selected for, funding to deliver KIU! through current HIV testing programs. Because CBOs do not have the capacity to host the technology, KIU! is hosted centrally at Northwestern and deployed by local CBO staff. Training and coaching of CBO staff on how to use the technology and integrate it into routine HIV testing is provided through capacity building assistance.

Strategy 2 is a "direct-to-consumer" (DTC) model where centralized staff at Northwestern primarily recruit participants nationally through online advertising campaigns and manage participants' engagement. HIV/STI testing kits are shipped directly to YMSM, and upon documentation of an HIV negative test result, participants are delivered KIU! This DTC model has been widely adopted in technology dissemination but it is innovative in terms of implementation of HIV prevention programs.

The investigators will accomplish the goal to inform eHealth intervention implementation with two specific aims:

Aim 1: Compare two implementation strategies using a cluster randomized trial.

The investigators selected a type III hybrid implementation-effectiveness design, which prioritizes empirical comparison of implementation strategies while also collecting evidence of effectiveness. The 75 counties with the largest YMSM populations will be randomized 2:1 to either Strategy 1 (CBO) or Strategy 2 (DTC). KIU! will be delivered in each county for two years. In this cluster randomized trial, the unit of randomization is the county, with implementation and effectiveness outcome data collected from YMSM participants, CBOs, and the technology vendor. The primary outcomes are public health impact (defined as reach X effectiveness) and cost per infection averted, a key component of CDC decision making about which HIV prevention programs will be supported.

Aim 2: Examine adoption characteristics that explain variability in implementation outcomes.

There is remarkable variability in implementation success when CBOs adopt evidence-based interventions. The investigators will seek to explain variability in implementation success across counties by conducting mixed-methods research on the domains from the consolidated framework for implementation research (CFIR), such as county characteristics, adaptations, support from organization leadership, and approach to planning adoption. Data on CFIR characteristics will be collected through administrative data, surveys, and telephone interviews with key stakeholders.

In addition to these 2 specific aims, the investigators will explore sustainment of KIU! at the completion of the study. CBOs will be provided with materials to facilitate applying for external funding to continue to provide KIU! after study completion, including an Impact Tool to estimate local impact and costs. The investigators will also examine factors that predict applying for funding and ongoing sustainment. For Strategy 2, through consultation with the CDC, capacity building assistance providers, health departments, and small businesses involved in Health 2.0 initiatives, the investigators will examine models for ongoing sustainment of the DTC model.

Study Type  ICMJE Interventional
Study Phase  ICMJE Not Applicable
Study Design  ICMJE Allocation: Randomized
Intervention Model: Parallel Assignment
Masking: None (Open Label)
Primary Purpose: Prevention
Condition  ICMJE
  • Gonorrhea
  • Chlamydia
Intervention  ICMJE Behavioral: Keep It Up! 3.0
KIU! is an online HIV prevention intervention developed for high-risk young men who have sex with men (YMSM) who recently tested HIV negative. Content was developed in collaboration with YMSM-serving CBOs and subjected to usability testing with diverse YMSM. The Information-Motivation-Behavioral Skills model and eLearning principles guided development of highly interactive, engaging, and culturally relevant health messages. KIU! involves 7 modules completed across 3 sessions done at least 24 hours apart, totaling ~1 hour of main content, plus 2 booster sessions at 3- and 6-month follow-ups. Each module is based on a setting or situation relevant to YMSM (e.g., connecting to the gay community and meeting guys through apps), with developmentally appropriate behavior change content embedded. KIU! uses diverse delivery methods (e.g. videos, animation, games) to address HIV knowledge gaps, motivate safer behaviors, teach behavioral skills, and instill self-efficacy for preventive behaviors.
Other Name: KIU! 3.0
Study Arms  ICMJE
  • Experimental: Community Based Organization (CBO) Delivery

    CBOs will be selected through a Request for Proposal (RFP) process. The RFP will describe research and KIU! delivery activities to be conducted as part of the study and allowable budget to carry out the activities. The RFP will ask CBOs to describe past experience providing HIV services for YMSM. Selected CBOs will recruit participants into the intervention and encourage participants to complete each session of intervention content at baseline, 3 Month Follow-up, and 6 Month Follow-up.

    Participants in the CBO delivery arm will receive baseline HIV and STI testing at a CBO. Participants will receive an at-home STI test kit at 12 Month Follow-up.

    Intervention: Behavioral: Keep It Up! 3.0
  • Experimental: Direct to Consumer (DTC) Delivery

    In the DTC arm, participants will be recruited online via paid social media advertising (e.g., Facebook, Instagram). Advertisements will target placements by age, gender, sexual orientation, racial background, "likes" that are relevant to YMSM (e.g., local LGBT organizations, "out" celebrities), and location (i.e., target county). Dating/sex-seeking apps (e.g., Grindr) will also be used to recruit YMSM, using a similar advertising approach as social media. These online recruitment strategies will be supplemented by referrals from local organizations and participant registries, and snowball recruitment. Study staff at Northwestern University will manage this recruitment process and encourage participants to complete each session of intervention content at baseline, 3 Month Follow-up, and 6 Month Follow-up.

    Participants in the DTC delivery arm will receive at-home HIV and STI test kits at baseline. Participants will receive an at-home STI test kit at 12 Month Follow-up.

    Intervention: Behavioral: Keep It Up! 3.0
Publications * Not Provided

*   Includes publications given by the data provider as well as publications identified by Identifier (NCT Number) in Medline.
Recruitment Information
Recruitment Status  ICMJE Recruiting
Estimated Enrollment  ICMJE
 (submitted: March 28, 2019)
Original Estimated Enrollment  ICMJE Same as current
Estimated Study Completion Date  ICMJE May 2023
Estimated Primary Completion Date May 2023   (Final data collection date for primary outcome measure)
Eligibility Criteria  ICMJE

Inclusion Criteria:

  • recently received an HIV negative test result OR self-report negative or unknown HIV status
  • had condomless anal sex with another man in the prior 6 months
  • resides in an eligible county
  • speaks/reads English
  • has an active email address

Exclusion Criteria:

  • has not been on PrEP and adherent for 6 months prior
Sex/Gender  ICMJE
Sexes Eligible for Study: Male
Gender Based Eligibility: Yes
Gender Eligibility Description: Cisgender men
Ages  ICMJE 18 Years to 29 Years   (Adult)
Accepts Healthy Volunteers  ICMJE Yes
Contacts  ICMJE
Contact: Brian Mustanski, PhD 312.503.5421
Contact: Krystal Madkins, MPH 312.503.6529
Listed Location Countries  ICMJE United States
Removed Location Countries  
Administrative Information
NCT Number  ICMJE NCT03896776
Other Study ID Numbers  ICMJE R01MH118213( U.S. NIH Grant/Contract )
Has Data Monitoring Committee Yes
U.S. FDA-regulated Product
Studies a U.S. FDA-regulated Drug Product: No
Studies a U.S. FDA-regulated Device Product: No
IPD Sharing Statement  ICMJE
Plan to Share IPD: Yes
Plan Description: The PI agrees to develop a de-identified database, codebook, and mechanism by which IPD could be shared with other investigators upon approval of the PI. Data will be available for request approximately 6 months after completion of the project. Interested investigators will be asked to complete a standardized request form stating the specific aims of the analysis, the analytic plans, resources the requestors have to carry out the project, the proposed timeline, and distribution goals (manuscripts and/or grant application). The PI will review these requests to determine whether the proposed analyses constitute an innovative and significant exploration of the data, whether the proposed team has sufficient resources to undertake the request, how data will be protected/managed, and whether there are sufficient resources to honor the request. If any of these issues are problematic, the PI will attempt to negotiate a fair resolution with the interested parties and/or with NIH program staff.
Responsible Party Brian Mustanski, Northwestern University
Study Sponsor  ICMJE Northwestern University
Collaborators  ICMJE
  • National Institute of Mental Health (NIMH)
  • Boston Medical Center
  • Weill Medical College of Cornell University
  • Marquette University
Investigators  ICMJE
Principal Investigator: Brian Mustanski, PhD Northwestern University
PRS Account Northwestern University
Verification Date July 2020

ICMJE     Data element required by the International Committee of Medical Journal Editors and the World Health Organization ICTRP