The Ethics of AI Voice Cloning for Family Memory

In 2024, voice cloning became a consumer capability. Feed a model five minutes of clean audio and it can say anything, in any language, forever. In the two years since, a category of services has emerged around "AI ancestors" — tools that let families speak with an AI version of a deceased loved one, built from recordings made in life.

We build one of these services. We have also argued publicly against how most of them are being built. This is the honest version of where we land on the ethics — and what a responsible implementation looks like.

The three things that could go wrong

1. Consent that didn't happen

Most grandparents alive today never imagined their voice could be reconstructed from an old home video. They didn't opt into voice cloning because the concept didn't exist when they recorded. The default question, then, is: who gets to decide whether to clone them now?

In almost every voice-cloning service currently on the market, the answer is: whoever has the recordings. That's not ethics — that's access. A son who has his father's voicemails doesn't have his father's consent.

2. Hallucination passed off as memory

A large language model attached to a voice can say anything that sounds plausible. If you ask your AI-reconstructed father "what did you think when I told you I was marrying Jennifer?", the system will produce a fluent, emotionally-satisfying answer. It may be based on real source material. It may be confabulated. Most current services don't distinguish.

The result: you may end up grieving an AI's opinion of your father, not your father's opinion.

3. Commercial and political use

If a voice model exists, it can be leaked, stolen, sold, or subpoenaed. A voice used by a grieving daughter today could be used by a scammer tomorrow, a campaign the year after. Services that don't design for this are offloading a real risk onto families who don't fully understand it.

What a responsible service looks like

Consent is explicit, recorded, and revocable

The subject — the person whose voice is being cloned — must themselves record a consent statement. Not the adult child. Not the estate. The subject. In their own voice, on the record, saying they consent.

That recording is stored with the model and displayed to anyone who invokes it. The subject can revoke at any time. If the subject dies without ever recording consent, voice cloning is not permitted, no matter who asks.

Posthumous rules reflect lifetime wishes

If the subject consented during life, their voice model continues to function according to the exact scope they agreed to — family-only, private, no commercial use. If they opted out during life, no family member can override that. A death certificate is not a consent upgrade.

AI outputs are source-linked

Every generated sentence from a reconstructed voice links to a specific captured answer from the subject. When the question asked is outside what the subject ever talked about, the system says so — it does not make up an answer. "I don't know" is not a failure mode. It's a feature.

No training. Ever.

Your family's voice and content are not used to train any public AI model. Not ours, not our vendors', not a research consortium's. This is a contractual commitment that we can be sued over, not a marketing line.

Destruction on request

Any family member featured in a voice model can request permanent destruction of that model. We do it within 24 hours of verified identity. No arguments, no retention-policy arguments, no "but your siblings…" We do it.

The case for doing this at all

You might read the above and conclude the responsible answer is: don't build voice cloning at all. We considered this carefully. Here's why we disagreed.

Grieving families have, for the entirety of human history, preserved what they can. Letters. Diaries. Video. Photographs. Memorial portraits. An archive of voice — the closest thing to the person themselves — is a continuation of that long tradition, not a departure from it. The question isn't whether families will try to keep the voices of the people they love. The question is whether the tools available to them will be built responsibly, or irresponsibly.

We would rather be one of the responsible options than boycott the category and watch it be defined by the irresponsible ones.

The line we won't cross

"We will not build the dead-person chatbot some competitors are racing toward. An honest legacy experience is not a séance."

That's from our public Trust & Ethics document. It's not abstract — it's the specific design choice not to build a conversational AI that impersonates the subject answering novel questions as if they were alive. What we will build, and what we do build, is a voice-delivered archive of what the subject actually said, navigated by descendants who want to hear them.

The distinction matters. It's the difference between a library and a séance. Between a museum and a Disney-animatronic. Between memory and magic.

What we ask of you

If you're considering any voice-cloning service — ours or anyone else's — ask these five questions before you commit:

1. Did the subject themselves consent, on the record, before you started?
2. Can the subject (or their estate) revoke?
3. Are AI outputs linked to specific source material?
4. Is the voice model used for training any other system?
5. What happens to the model if the company is acquired or shuts down?

If a service can't answer all five cleanly, don't use it. Regardless of which service you choose, the questions above are the ones that will look back at you clearly ten years from now.