Q&A
Council member Bottcher questions OTI about city's small business chatbot accuracy and effectiveness
0:54:42
·
4 min
Council Member Erik D. Bottcher questions Alex Foard from the Office of Technology and Innovation (OTI) about the city's chatbot for small businesses, focusing on its accuracy, testing, and effectiveness following reports of incorrect information being provided. Foard provides limited information, citing improvements made to the chatbot and the need to consult with the specific team for more details.
- The chatbot had issues providing incorrect information about employee tips and discrimination
- OTI made enhancements to reduce errors, but specific quantification was not available
- Foard couldn't provide details on pre-launch testing or real-time data access for the chatbot
- The overall user experience with the chatbot is described as "generally quite positive"
Erik D. Bottcher
0:54:42
Hi.
0:54:44
Last fall, the city announced that you were releasing the first chatbot in the city's history that would assist small businesses around the city.
0:54:58
And in the spring, it was announced that it was reported that the chatbot was giving incorrect information to users the chatbot told some users that it was allowable to take a portion of their employee's tips the chatbot answered that it was allowable to discriminate on source of legal income.
0:55:33
Can you give us an update on the chatbot since all those stories came out in April, is the chatbot now giving accurate information?
0:55:47
Or are those issues persisting and what efforts are underway to address it?
Alex Foard
0:55:55
Sure.
0:55:55
Thanks for the question.
0:55:57
I will speak at a high level, not the business owner of that particular tool.
0:56:02
So happy to take back more specific questions for the team there.
0:56:07
There were enhancements that were made earlier this year, upgrading the models, etcetera, that reduced the number of errors and the instances of information given when it should not have been given, which is one of key issues.
0:56:21
Again, I don't have the specific quantification of that.
0:56:24
I can go back to the OTI team for more specifics there, but what we have found is a reduced set of errors and overall positive interaction between users and the chatbot.
Erik D. Bottcher
0:56:35
What kind of testing was done prior to the launch of the chatbot to ensure its accuracy?
Alex Foard
0:56:43
Alright.
0:56:43
That's the sort of detail that I'll need to go back to the team at OTI for.
Erik D. Bottcher
0:56:47
And do you based on just your knowledge of the technology, do you know what would cause a chatbot to give inaccurate answers?
Alex Foard
0:56:57
So, I I mean, I think the use of the term inaccurate answers is sort of an umbrella term.
0:57:02
Right?
0:57:02
That actually covers a couple different categories of output.
0:57:06
You know, there's whether or not it's responding to a question that is not supposed to respond to whether or not it's coming up with the right sequence of words.
0:57:15
So there's a dynamic set of of factors that could lead to that that kind of inaccurate output.
0:57:20
So, you know, these are are components of the technology that need to be understood and addressed and mitigated, which is why efforts were made to improve the performance of the tool earlier this year.
Erik D. Bottcher
0:57:33
With chat, GPT, and other tools, it often only contains information up to a certain point in time, sometimes even a couple years in the past.
0:57:45
Does the chatbot that the city is using?
0:57:48
Does I have access to, like, real time developments in and changes in city policy?
Alex Foard
0:57:55
I can't speak to the specific set of data on which the chatbot is either trained or currently pulling information.
0:58:02
The information that it pulls with respect to small business services which is in its intended output, of course, is confined to the set of data that it's supposed to report on.
Erik D. Bottcher
0:58:12
Based on what you do know about the chatbot, you feel comfortable saying that this is a tool that's effective for a small business operators in New York City.
0:58:24
And when they use it, they will be getting accurate answers to their questions.
Alex Foard
0:58:29
Yeah.
0:58:29
That's a great question.
0:58:30
I think one thing that we always wanna be sort of clear up on front that it's the direct business owners of tool, the ones who are using it, who really need to make that sort of definitive call on how effective is this?
0:58:40
Is it doing the thing that we needed to do?
0:58:42
So happy to take some more specific questions back for colleagues at OTI.
0:58:46
In general, the chatbot was developed and is presented to the public in a way that's consistent with our AI principles.
0:58:54
Transparent has language around its use and how it is intended to to be used, what the sorts of limitations it may have, etcetera.
0:59:01
And my understanding is that, again, user experience is generally quite positive.
