Robyn Hussa Farrell, Jared Minkel, Melissa Pish
White paper published Aug 30, 2024
Download pdf:
1. Overview of Approach to Engagement
Dozens of studies have now demonstrated that digital interventions can improve mental health (Horwitz et al., 2024; Taylor et al., 2024). However, these same tools suffer from high dropout rates which severely limits their usefulness in clinical practice (Boucher & Raiker, 2024, see Appendix A; Amagai et al, 2022). Furthermore, there is a demonstrated small or non-existent difference in effectiveness for those who engage in digital interventions. This lack of improvement persists across approaches, even in those where an adequate translation of real-world treatments to digital formats has occurred (Kambeitz-Ilankovic et al., 2022). Despite the effort to convert well-established therapies to digital methods, adoption rates remain low and digital interventions are often underutilized (Boucher & Raiker, 2024). That’s why Resiliency Technologies (RT) developed its array of Sharpen products to optimize engagement as well as effectiveness.
Key Elements of the Sharpen Approach:
RT products have been developed in collaboration with experts to maximize efficacy, but also with the products’ intended users. Our products are created “by-youth-for-youth” using robust listening and integrating user voices at every stage of product development.
Each product is enriched with state of the art, documentary style film clips that are short, digestible, and include models who represent the intended users in terms of age, ethnicity, gender and other aspects of identity
RT products always incorporate an interpersonal element which can be a clinician recommending a product or a teacher guiding a class through a course,
RT products are designed flexibly to deliver therapeutic value, even when used imperfectly (as is always the case in real world usage). Our modules enhance one another but also stand alone. We encourage users and clinicians to think of them as a string of single session interventions.
Within the original version of the Sharpen wellness platform (for example in each customized wellness app deployed to a college or community organization), each individual intervention could be accessed in isolation. Initial analyses indicated that these single-sessions were beneficial to the user. Once this was determined, the content was assembled into a sequence, with the first sequence aiming to de-stigmatize mental illness and treatment. We assembled them into a series intending that they have a sequence, with de-stigmatizing being the first priority (Hinshaw et al., 2024; Martinez & Hinshaw, 2016).
Participants have a self-discovery library available for additional learning after they have completed the first series of "prescribed" or recommended interventions.
A majority of students that were interviewed about the effectiveness of Sharpen indicate that the17-week SMHL course is an invaluable tool in maintaining and promoting their mental health (Levy et al, 2025).
2. Evidence of Engagement
Over the past decade, Resiliency Technologies (RT) has regularly deployed their Sharpen wellness interventions to various groups - from the parents of foster children to high school students to clinicians. Each deployment was conducted with the goal of a) improving the mental health and mental health literacy of the groups and b) collecting data to ensure the continued development of the Sharpen product. As a result, Sharpen has gathered a massive amount of actionable data over the initiation of 35 research studies. The data discussed in this document represents a brief analysis of two of RT’s larger studies; however, the information represents only a small component of the data available. Analysis remains ongoing in many of these studies. For additional information regarding ongoing studies and analysis, visit www.SharpenMinds.com/research.
Our scientific roadmap for establishing Sharpen’s status as the leader in engagement in digital therapeutics for adolescents and young adults includes real world studies to quantify 1) activation rates, defined as the number of users who at least minimally access the Sharpen modules 2) the level of engagement needed to achieve clinically meaningful improvement in responders and 3) the percentage of users who engage at that level. At present, we have qualitative and quantitative data from real world deployments, but are in the process of funding larger studies with more advanced methods.
Below we review mixed methods evidence (quantitative and qualitative) that strongly suggest our products are well liked by users (very high in “acceptability”) and highly engaging. Furthermore, qualitative responses and interviews illuminate the key features that differentiate our product from others in the industry.
3. Competitive Analysis
User Engagement
Since its inception in 2018, Sharpen has maintained a high level of engagement with its users. Between June 2019 and December 2023, the Sharpen platform consisted of 24,777 users, with 47,393 completed sessions. According to our Google Analytics, the average session length was 4 minutes and 45 seconds, suggesting a meaningful engagement with the product (Forbes et al., 2023). Within the first 7 months of 2024, Sharpen maintained a similar level of engagement with 4,965 users completing 9,094 sessions at an average length of 7 minutes and 14 seconds. While these statistics encompass all Sharpen users, specific subsets have provided a wealth of qualitative information.
The Sharpen course can be tailored to a unique set of users, depending on the intended target of the content. For example, Sharpen can be deployed as an overall Mental Health Literacy course, or the content can be tailored to provide education and resources on a specific mental illness, such as depression or eating disorders. At this time, several deployments of the Sharpen app within different subgroups have occurred, and both users and facilitators (physicians) have provided their feedback.
Collegiate Benefits
One comprehensive and long-term deployment occurred on the campus of the University of West Georgia, where students participated in a 17-week Sharpen course. At the conclusion of the course, students (n=109) were asked their opinion about the Sharpen product, and a total of n=101 students provided qualitative feedback for analysis. Overall, students who provided feedback responded with a favorable opinion of the app, and 95.92% of student feedback contained overwhelmingly positive statements.
A qualitative analysis of responses identified that the three most common strengths that were mentioned were:
The app was fun and easy (present in 19% of coded responses),
The content, including the videos, was relatable (present in 16% of coded responses),
The resources were valuable (present in 14% of coded responses).
Furthermore, students consistently reported additional strengths of the app, including the usefulness of the practical skills and tools, which served to increase their knowledge of mental health and reduce mental health stigma.
Multiple students indicated that Sharpen was an invaluable resource that they wished was more widely available:
“Overall I can honestly say as a 6 year student I wish this app would be mandatory or more well known for all students on campus. In the short month I have had it, not only have I been educated and have my eyes opened. I even practice these things with my family members, and it really helps. It makes you more conscious and mindful of yourself and other people around which makes you a better person.”
“I definitely recommend the UWG Sharpen app to anyone who really wants to get in tune with themselves. It has helped me, and I will be continuing to use the app past this course.”
“I believe that the activities and tips this app has are some of the best resources a student can access, and we luckily get them for free.”
“... it [the Sharpen course] has helped me find the calm within the chaos and I'm eternally grateful for you (and your team) . Thank you, thank you, THANK YOU!!!!!”
Ultimately, students at the University of West Georgia positively endorsed the course, with several students indicating that it was a resource that they would continue to utilize in the future. A full analysis of the 17-week SMHL course is being led by Dr. Duke Biber, Jennifer Levy and Julianna Fassass, with anticipated completion in Spring of 2025.
Clinical Benefits
Beyond collegiate engagement, Sharpen courses have proven beneficial in clinical settings. Sharpen courses were deployed within a prominent New York Health System to assist clinicians in learning more about suicide prevention. Of the providers who took the course, 90% were strongly satisfied with the training and would recommend that other providers take the course as well. This engagement reflected high confidence that the material would be clinically beneficial as providers reported an increased comfort and confidence level in providing care for individuals expressing suicidal tendencies, as evidenced by the statistically significant increase in mean scores in Table 1.
Results
Both population and clinical level analysis indicate that Sharpen is a uniquely beneficial platform. The general public and providers confirm that the Sharpen platform is straightforward, comprehensive, and engaging, and many users will continue to engage with the content and recommend Sharpen to others.
Appendix
A. Summary of Evidence for Barriers and Facilitators to Engagement with Digital Mental Health Interventions
Appendix A: (Boucher & Raiker, 2024)
B. Ongoing studies
Resiliency Technologies is continuously researching and innovating through their various deployments of the products. Additional and ongoing studies can be found on our website and include:
Foster Parent Resilience: A study conducted for the South Carolina Department of Social Services which utilized Sharpen’s mental health literacy and MBSR course to support foster parent self-care and assess the impact of the program.
Sharpen Eating Disorder Prevention Training for Physicians: A feasibility study analyzing Sharpen Eating Disorder Awareness & Prevention training program for physicians is currently underway, funded through support from Spartanburg Regional Healthcare Foundation and Edward Via College of Osteopathic Medicine.
Sharpen Mental Health Literacy (SMHL) for suicide prevention in medical students: Analysis is ongoing in 3 year SAMHSA grant when Sharpen has been implement as a suicide prevention toolkit and training platform for medical students.
While the list contained within this paper is not comprehensive, we believe it demonstrates the wide ranging impact of the product and highlights the potential for activation and engagement across groups. To view additional studies, please visit our website.
Acknowledgements
Resiliency Technologies gratefully acknowledges our longstanding partnership with the University of West Georgia for their five-year collaboration on Sharpen, which has been instrumental in supporting college student and athlete mental health literacy. We also extend our sincere appreciation to VNS Health (New York City) for their valuable collaboration on NY Office of Mental Health-funded suicide prevention efforts focused on LGBTQIA+ youth. These partnerships have been fundamental to advancing our understanding and implementation of mental health support services.
References Cited
Amagai, S., Pila, S., Kaat, A. J., Nowinski, C. J., & Gershon, R. C. (2022). Challenges in Participant Engagement and Retention Using Mobile Health Apps: Literature Review. Journal of medical Internet research, 24(4), e35120. https://doi.org/10.2196/35120
Boucher, E., Raiker, J. (2024). Engagement and retention in digital mental health interventions: a narrative review. BMC Digit Health 2,52. https://doi.org/10.1186/s44247-024-00105-9
Forbes, A., Keleher, M. R., Venditto, M., & DiBiasi, F. (2023). Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review. Journal of medical Internet research, 25, e43727. https://doi.org/10.2196/43727
Hinshaw, S. P., Porter, P. A., & Ahmad, S. I. (2024). Developmental psychopathology turns 50: Applying core principles to longitudinal investigation of ADHD in girls and efforts to reduce stigma and discrimination. Development and psychopathology, 1–15. Advance online publication. https://doi.org/10.1017/S0954579424000981
Horwitz, A. G., Mills, E. D., Sen, S., & Bohnert, A. S. (2024). Comparative effectiveness of three digital interventions for adults seeking psychiatric services: A randomized clinical trial. JAMA Network Open, 7(7), e2422115-e2422115.
Kambeitz-Ilankovic, L., Rzayeva, U., Völkel, L., Wenzel, J., Weiske, J., Jessen, F., Reininghaus, U., Uhlhaas, P. J., Alvarez-Jimenez, M., & Kambeitz, J. (2022). A systematic review of digital and face-to-face cognitive behavioral therapy for depression. NPJ digital medicine, 5(1), 144. https://doi.org/10.1038/s41746-022-00677-8
Kim, J., Aryee, L. M. D., Bang, H., Prajogo, S., Choi, Y. K., Hoch, J. S., & Prado, E. L. (2023). Effectiveness of Digital Mental Health Tools to Reduce Depressive and Anxiety Symptoms in Low- and Middle-Income Countries: Systematic Review and Meta-analysis. JMIR mental health, 10, e43066. https://doi.org/10.2196/43066
Levy, Biber & Fassas. 2025 [Forthcoming]. A Program Evaluation of the Sharpen® Mental Health Literacy (SMHL) College Course at a Southeastern University in the United States.
Martinez, A. G., & Hinshaw, S. P. (2016). Mental health stigma: Theory, developmental issues, and research priorities. In D. Cicchetti (Ed.), Developmental psychopathology: Risk, resilience, and intervention (3rd ed., pp. 997–1039). John Wiley & Sons, Inc.. https://doi.org/10.1002/9781119125556.devpsy420
Taylor, M., Liu, M., Abelson, S. et al. (2024). The Reach, Effectiveness, Adoption, Implementation, and Maintenance of Digital Mental Health Interventions for College Students: A Systematic Review. Curr Psychiatry Rep 26, 683–693. https://doi.org/10.1007/s11920-024-01545-w
Commentaires