Corporate mental health provider rolls back AI chatbot

Jim “the AI Whisperer” Nightingale said even without the extreme responses he spurred, mental health advice was not a good use for generative AI. (Image: Getty)

Ben Moore

Thu, 17 Oct 2024

A startup whose customers include major employers Spark, Contact Energy and two banks has rolled back an errant artificial intelligence chatbot that was willing to offer one independent researcher graphic advice on how to perform incest.The chatbot, part of a platform provided by Auckland startup Clearhead, also counts IRD, insurer UniMed, TSB and The Co-operative Bank and others among its customers.Clearhead’s main product is a software platform that enables businesses to offer therapy sessions to their staff and access a wider library o...

Subscribe to BusinessDesk

Stay informed on business, government and financial developments across New Zealand.

Deeply researched, twice-edited and fact-checked news
Annual subscribers also receive a complimentary subscription to The Wall Street Journal
Personalised email news alerts, plus gift up to 5 stories a month to non-subscribers

You can cancel anytime with two clicks, an email or a phone call.
Find out more.

We are serious about journalism.

Already a Subscriber ? Sign in here.

Name {{ registerForm.errors.get('name') }}

Email address {{ registerForm.errors.get('email') }}

Phone Number {{ registerForm.errors.get('phone') }}

Job Title {{ registerForm.errors.get('title') }}

Password Minimum password length of 8 characters.
Require at least one upper and lowercase, numeric, and special character. {{ registerForm.errors.get('password') }}

Repeat password {{ registerForm.errors.get('password_confirmation') }}

We had trouble validating your card. It's possible your card provider is preventing us from charging the card. Please contact your card provider or customer support.

Credit Card

Cardholder's name

{{ __(plans[index].name) | capitalize }} {{ plans[index].attributes.old_price | currency }} {{ plans[index].price | currency }}

All subscriptions auto renew but are easy to cancel.

I Accept The Terms Of Service

{{ registerForm.errors.get('terms') }}

Tax: {{ taxAmount(selectedPlan) | currency }}

Total Price Including Tax: {{ priceWithTax(selectedPlan) | currency }} {{ selectedPlan.type == 'user' && spark.chargesUsersPerSeat ? '/ '+ spark.seatName : '' }} {{ selectedPlan.type == 'user' && spark.chargesUsersPerTeam ? '/ '+ __('teams.team') : '' }} / {{ __(selectedPlan.interval) | capitalize }}

Markets

2 Cheap Cars signals branding gear change to chase Turners

At the AGM, a shareholder cited Turners' “very aggressive marketing strategy”.

Gregor Thompson 12:00pm

The Quiz Free

QuiznessDesk, Friday, Sept 26

Think you're clever enough?

The Quizmaster 9:15am

Technology

Govt agency defends support for vanishing venture capitalist

NZ Growth Capital Partners refuses to detail problems with Hillfarrance startup fund.

Maria Slade 5:00am

Sponsored

AI agents: opportunity or graveyard? Spark’s five-point playbook for success

Artificial intelligence agents – autonomous digital entities that analyse data, automate decisions, and act independently – are set to drive a new wave of business transformation.

Spark Enterprise

Sponsored

Future leaders start by Dreaming Big

How OnBeingBold empowers Year 13 girls at free leadership events.

OnBeingBold

More Technology

Technology

Govt agency defends support for vanishing venture capitalist

NZ Growth Capital Partners refuses to detail problems with Hillfarrance startup fund.

Maria Slade 5:00am

Law & Regulation

Rich listers’ rescue of Ubco: Sale price revealed

Three rich-list families, plus the multinational Jubilee Investments, paid $5 million for Ubco, the latest receivers’ report reveals.“The transaction included the transfer of shares in Ubco subsidiaries, intellectual property, goodwill, plant and equipment, stock, and certain con...

Staff reporters 25 Sep 2025