An AI chatbot launched by the New York Metropolis authorities to assist enterprise homeowners entry pertinent data has been spouting falsehoods, at instances even misinforming customers about actions which are in opposition to the regulation, in response to a report from The Markup. The report, which was co-published with the native nonprofit newsrooms Documented and The Metropolis, consists of quite a few examples of inaccuracies within the chatbot’s responses to questions regarding housing insurance policies, employees’ rights and different matters.
Mayor Adams’ administration launched the chatbot in October as an addition to the MyCity portal, which launched in March 2023 as “a one-stop store for metropolis companies and advantages.” The chatbot, powered by Microsoft’s Azure AI, is aimed toward present and aspiring enterprise homeowners, and was billed as a supply of “actionable and trusted data” that comes immediately from the town authorities’s websites. However it’s a pilot program, and a disclaimer on the web site notes that it “could often produce incorrect, dangerous or biased content material.”
In The Markup’s checks, the chatbot repeatedly offered incorrect data. In response to the query, “Can I make my retailer cashless?”, for instance, it replied, “Sure, you may make your retailer cashless in New York Metropolis” — even though New York Metropolis banned cashless shops in 2020. The report reveals the chatbot additionally responded incorrectly about whether or not employers can take their employees’ ideas, whether or not landlords have to simply accept part 8 vouchers or tenants on rental help, and whether or not companies have to tell workers of scheduling modifications. A housing coverage professional that spoke to The Markup known as the chatbot “dangerously inaccurate” at its worst.
The town has indicated that the chatbot continues to be a piece in progress. In an announcement to The Markup, Leslie Brown, a spokesperson for the NYC Workplace of Expertise and Innovation, stated the chatbot “has already offered hundreds of individuals with well timed, correct solutions,” however added, “We are going to proceed to give attention to upgrading this device in order that we are able to higher help small companies throughout the town.”