Q&A
Data collection and evaluation of mental health programs in schools
0:32:56
·
4 min
Council Member Linda Lee inquires about data collection and evaluation methods for mental health programs in schools. Dr. Erica Lynne Smith from DOHMH explains their approach to measuring program uptake and implementation success.
- DOHMH collects utilization data, such as the number of services provided and number of attendees
- No personal health information or treatment data is collected
- The focus is on ensuring that programs are being utilized by students
- Dr. Smith emphasizes the importance of implementation support in program success
- DOHMH has a team that works closely with 548 schools to monitor program uptake and address any issues with referral processes
Linda Lee
0:32:56
Okay.
0:32:56
And how are you yeah.
0:32:57
Because I'd love to know.
0:32:58
Do you know are you able to speak to how like, how's that data being collected
Dr. Erica Lynne Smith
0:33:03
since it's a new program?
0:33:05
I think, again, I I don't have data around it.
0:33:08
We would know super it's a super small initiative right now.
0:33:16
You know, our office, you know, keeps track of who attends, but, you know, though that's huge or the numbers, very aggregate numbers on on who would attend adult mental health first aid.
0:33:28
We don't collect any identifying information about that.
0:33:31
Again, we would have to speak, you know, in more detail about what would be appropriate in terms of implementation and and privacy around, with our colleagues and with city council.
Linda Lee
0:33:43
Okay.
0:33:43
Because I think on the data collection piece, also, I'd like to see if you could let us know what, metrics you are are using to also then later evaluate if it was successful or not, which I guess go hand in hand.
0:33:56
But
Dr. Erica Lynne Smith
0:33:56
Successful is kind of a, I try to stay away from the word successful and really more about, you know, was there uptake or not?
0:34:07
Subject it's subjective.
0:34:08
Right?
UNKNOWN
0:34:08
No.
Dr. Erica Lynne Smith
0:34:08
No.
0:34:08
Yeah.
0:34:09
But so was there uptake or not?
0:34:11
So when we work with, providers to provide services in schools, we collect utilization data.
0:34:17
So it's there's no PHI.
0:34:18
There's no treatment data.
0:34:20
There's no Mhmm.
0:34:20
We really are involved with ensuring that, very colloquially, if we're selling something and they're not buying it, then it doesn't matter.
0:34:30
It can be the best intervention in the world.
0:34:31
So we collect, you know, number of services provided, number of attendees, and that's really it in terms of the whenever we partner with with
Linda Lee
0:34:42
And then is there any feedback?
0:34:43
Because I I agree.
0:34:44
I think in terms of maybe effective is a better word.
0:34:46
I don't know what the right word is.
0:34:48
But, to your point, I think the utilization piece is really important because if it's because then if the numbers are low and students are not utilizing these services, then is it a question of that they don't know that it's there?
0:35:01
Is it a question that, you know, they don't know about the trainings or haven't been informed enough?
0:35:06
Or is it that they're just not interested?
0:35:08
Or maybe this is not the right space or the, you know, medium for it?
0:35:11
I don't know.
Dr. Erica Lynne Smith
0:35:11
Right.
0:35:12
And I've seen many programs come and go.
0:35:13
I've been working in this field since 911, actually, in a school.
0:35:17
And, you know, I I I caution that implementation is often neglected, and it's so important because, a lot of times what happens is when you reflect on something that, quote, unquote, didn't work, it's because of the implementation support wasn't there.
Linda Lee
0:35:37
Right.
Dr. Erica Lynne Smith
0:35:38
Not that it's not an effective program, but, again, implementation is about on the ground and exactly what you're mentioning.
0:35:46
Are people utilizing the service, and is it appropriate to the population it intends to serve?
0:35:54
Right.
Linda Lee
0:35:54
And I guess, in that same vein, I'd be wondering what, DOH, MH, and DOE's, sort of, outreach process is, given the different languages, cultural, you know, nuances, and the fact that a lot of folks are simply just not gonna talk about this no matter what.
0:36:17
And so how do we do the outreach?
Dr. Erica Lynne Smith
0:36:19
So in the 548 schools that we're in, I can speak to that because we have someone on this on-site in those schools.
0:36:25
My the the teams that I have, my the my colleagues that are in schools have a caseload of 15 schools.
0:36:31
They know their schools.
0:36:33
They know their principals.
0:36:34
They know the providers that are on-site, and we can look at using that data if there's blips.
0:36:40
And what I mean by a blip is if there aren't any referrals, we can speak both to the DOE or I'm sorry.
0:36:48
New York City Public Schools, staff who are on-site, and we can also speak to the providers and really come together to really understand if there's not a you know, if the referral process is working, and that's what really sustains programming.
0:37:04
Having that sort of conduit through which both parties can come and talk about their concerns or things that aren't working, and then we can fix it.
