My workplace has a chatgpt-style bot that you can talk to using the regular workplace chat program. I guess it's primarily for asking questions about work policies and procedures, but I actually tried doing that a few times... the first time it spent 10 minutes trying to answer "What can you do?" and ultimately told me "I'm sorry, there's been a problem. I've let my manager know."
I decided to try to ask it something easier. "When is our next work holiday?"
It acts like it's typing for 5 minutes, then comes back with, "Did you mean [random employee's name]? Here is their contact information:" and then prints their entire contact card including which worksite they're at and who their manager is.
Their name did not contain, nor was it a typo of, anything in my question.
A coworker decided to ask the bot "Who is your manager?" And it was the only question it was able to answer straight away, by giving another employee contact card. Thing is, because of previous results, it's unclear if that is even accurate information.
And somebody decided this was useful enough to install as enterprise software to a global corporation.
anyway I'm about to quit that job in a few minutes