CMHA Toronto is leading a collaborative research initiative aimed at enhancing the capacity of community mental health agencies to measure service outcomes. The project’s ultimate goal is to provide higher quality care across the sector.
Frank Sirotich, CMHA Toronto’s Chief Research Officer, explains that this important and innovative study, begun in 2021, is “piloting performance indicators from standardized data that are used regularly by the sector so that we have common metrics to measure our outcomes.”
Frank and his team – researchers from CMHA Toronto, CMHA Ontario, Ontario Health and the Centre for Addictions and Mental Health (CAMH) – are working with 17 other participating agencies on the initiative. He emphasizes that it’s important that community mental health providers lead this research. “Their work in the community equips them with a unique understanding of the social, cultural and environmental contexts in which services are delivered. This understanding is crucial in ensuring that outcome measures are not only relevant but also feasible for all providers across the province to adopt,” he says.
Frank adds that what's happened in the province periodically is that new, standardized datasets have been created “but there hasn't been much investment in developing measures that come from that data or in using the data for planning and for quality improvement.”
The goal is to demonstrate our impact, identify where there are gaps that would help inform planning and advocate for investments where there are disparities.
Creating a focus on unmet needs to inform programming and advocacy
The study has developed a set of performance indicators derived from the Ontario Common Assessment of Need (OCAN), a standardized needs assessment instrument widely used in the community mental health sector. Several cycles of reports with these performance indicators have been generated to enable participating organizations to measure their own outcomes, do an apple-to-apple comparison with other service providers, and identify gaps in programming within individual organizations and across the community mental health sector.
These data led the research team to home in on a set of client needs known as “The Big Four,” which are emerging as “fairly consistently unmet,” says Frank. They are:
- Psychological distress (e.g., symptoms of anxiety and depression)
- Daytime activity (the presence or lack of structure provided by employment, education, community programs, volunteering, etc.)
- Company (social connectedness or the reverse, social isolation)
- Physical health
“What we think is particularly noteworthy is that these unmet needs go beyond the core competencies of many individual community mental health programs, such as case management,” Frank says. “Typically, these needs require a multidisciplinary approach and are addressed by services like counselling and therapy, supported employment programs, peer support programs, and primary care in the case of physical health.”
These may not be areas that some agencies have the programming, resources or focus to address, “so what it potentially speaks to is a need to think about integrating services across organizations so that our services best meet individuals’ needs of,” Frank adds.
Opportunities for cross-referrals and integration
To meet the multi-faceted needs of clients, agencies can pool their assets and services to address common areas of persistent unmet need identified through these performance indicators.
While CMHA Toronto is fortunate to have a variety of programs under its overall umbrella and can make relatively seamless referrals from one service to another, smaller agencies serving more targeted populations or issues may not have the same breadth or flexibility.
These organizations can work in partnership with other community mental health providers to address service gaps. “By reviewing these performance indicators collectively, community mental health providers can identify common areas of unmet need and then work in partnership with other organizations to pool resources and create clinical pathways that increase access to programming, addressing areas of unmet need,” Frank says. “Where needed programming doesn’t exist or has long wait times, the indicators can be used to advocate for investments by funders to close service gaps.”
Integrating or creating pathways to service, irrespective of the organization, can address unmet needs in a timelier fashion and is likely to reduce the individual's time in the service.
By using these measures to inform planning and to integrate services across providers, client needs can be addressed more quickly. This option for greater efficiency – increasing the speed of access to service and reducing the overall amount of time a client is involved with that service – is a welcome prospect in times of acute service shortages and long waitlists.
Getting inputs from those working ‘on the ground’
While top-down work to address these issues has been happening through the Roadmap to Wellness, Ontario Health’s mental health and addictions strategy, it's the first time that the community mental health sector in Ontario, and likely in Canada according to Frank, has come together to provide their own ‘on-the-ground’ point of view on measuring service outcomes to inform policy and planning – and, perhaps most importantly, make a data-informed case for specific investments.
The involvement of CMHA Ontario is key. CMHA Ontario is actively involved in the research and plays a role communicating the study’s findings to Ontario Health and the Mental Health and Addictions Centre of Excellence. It’s a channel that goes both ways: “CMHA Ontario’s more intimate knowledge of the needs and priorities of government lets us have that in mind, along with the needs of the providers, when undertaking this research,” Frank says.
Building capacity in community mental health and addictions agencies
The participatory research model being used, which has included stakeholders in shaping the study and seeks their ongoing advice throughout the project, isn't new but it is best practice.
Participating organizations are supported to analyze their own data and encouraged to share their findings with their staff, clients, and family members. This is meaningful for the agency itself and offers the potential for innovative and practical applications for service enhancements “because it's a bit of a different lens that’s applied,” says Frank.
“Some organizations have very limited internal decision-support capacity to be able to pull their own data and create reports. And even those organizations that do have some in-house capacity, they don't have external benchmarks,” says Frank. “This study is providing their own data back to them in a more usable form, with provincial comparators so they can see how their services compare to other similar services across the province and also how similar or different the clients they're serving are compared to others.”
In the big picture, the work is enabling community mental health organizations to be transparent and accountable to their stakeholders and funders. “I think that can enhance trust in and credibility of the sector,” says Frank. By demonstrating the sector’s knowledge and capability, it also builds a firm foundation for a case for support.
“At the end of the day, [the information from this research] will help providers tell their story to funders, clients, and the broader community.”
The next step is to “try to identify variables that are associated with or predict either better or worse outcomes,” says Frank. The researchers are now focused on factors that might predict persistent unmet need in a particular area or among certain client groups. “If we can identify these factors – they might be demographic, clinical, environmental or contextual – then providers can intervene earlier to try to mitigate that risk,” Frank explains.
Stay tuned for more from Frank and his team as the research progresses.
More News from CMHA Toronto
TCCS’s 5 Key Principles: Serving clients and communities better with compassionate, client-centred crisis care
Through vivid case examples, learn how TCCS’s non-carceral, compassionate, and client-centric approach puts “a foot in that revolving door of crisis to get to the root of the issue” for
Learn More
Creating a space for safety and support
Archana Lingendran, manager, and Matthew Carrol, clinical lead, of Toronto Community Crisis Service (TCCS) spoke with us about the pilot program launched July 2022 that is creating space and safety
Learn More
Overdose Awareness Day – A Day of Reflection and Action
August 31st marks International Overdose Awareness Day (IAOD), a day that signifies lives profoundly affected by the global overdose epidemic.
Learn More
Contact Us
West Office
700 Lawrence Ave W,
Suite #480 Toronto,
ON M6A 3B4
East Office
1200 Markham Rd,
Suite #500 Scarborough,
ON M1H 3C3
Subscribe to
Living. Working. Belonging.
A monthly newsletter from CMHA Toronto about thriving together and improving mental health care