Corporate mental health provider rolls back AI chatbot

Jim “the AI Whisperer” Nightingale said even without the extreme responses he spurred, mental health advice was not a good use for generative AI. (Image: Getty)

Ben Moore

Thu, 17 Oct 2024

A startup whose customers include major employers Spark, Contact Energy and two banks has rolled back an errant artificial intelligence chatbot that was willing to offer one independent researcher graphic advice on how to perform incest.The chatbot, part of a platform provided by Auckland startup Clearhead, also counts IRD, insurer UniMed, TSB and The Co-operative Bank and others among its customers.Clearhead’s main product is a software platform that enables businesses to offer therapy sessions to their staff and access a wider library o...

Unlock all stories free for 10 days

Stay informed on business, government and financial developments across New Zealand.

Deeply researched, twice-edited and fact-checked news
Annual subscribers also receive a complimentary subscription to The Wall Street Journal
Personalised email news alerts, plus gift up to 5 stories a month to non-subscribers

We won’t charge for 10 days. You can cancel anytime with two clicks, an email or a phone call. Find out more.

We are serious about journalism.

Name {{ registerForm.errors.get('name') }}

Email address {{ registerForm.errors.get('email') }}

Phone Number {{ registerForm.errors.get('phone') }}

Job Title {{ registerForm.errors.get('title') }}

Password Minimum password length of 8 characters.
Require at least one upper and lowercase, numeric, and special character. {{ registerForm.errors.get('password') }}

Repeat password {{ registerForm.errors.get('password_confirmation') }}

We had trouble validating your card. It's possible your card provider is preventing us from charging the card. Please contact your card provider or customer support.

Credit Card

Cardholder's name

{{ __(plans[index].interval) | capitalize }} {{ plans[index].attributes.old_price | currency }} {{ plans[index].price | currency }}

You will be charged in 10 days unless you cancel beforehand.
We will remind you three times before charging you.
All subscriptions auto renew but are easy to cancel.

I Accept The Terms Of Service

{{ registerForm.errors.get('terms') }}

Tax: {{ taxAmount(selectedPlan) | currency }}

Total Price Including Tax: {{ priceWithTax(selectedPlan) | currency }} {{ selectedPlan.type == 'user' && spark.chargesUsersPerSeat ? '/ '+ spark.seatName : '' }} {{ selectedPlan.type == 'user' && spark.chargesUsersPerTeam ? '/ '+ __('teams.team') : '' }} / {{ __(selectedPlan.interval) | capitalize }}