Really appreciated that the National Communication Museum AI Summit (Thanks, Emily Siddons) brought together a privacy professional (Patricia Calabro), a psychiatrist/science-fiction writer (Grace Chan https://lnkd.in/gXtd6N-k) and me, an AI engineer/scientist, to discuss trust and automated decision making.

Before the panel, I wandered through the museum and unexpectedly went down memory lane: childhood computers, old communication systems, and early internet culture. Here you can see me talking on an old rotary phone to John Sullivan, co-inventor of Wi-Fi, on the other side, and typing “hello from the future, CSIRO” into a simulated MSN Messenger chat room.

The panel and audience questions didn’t disappoint. One of the points I tried to make is that trust in AI is not just about evaluation metrics or evidence. Those things matter enormously, but humans do not trust purely rationally. We routinely over-trust and under-trust things despite evidence. Trust is deeply social and psychological.

That becomes especially important with modern AI systems because they can mimic empathy, vulnerability, confidence, personality, and even intimacy. Humans are wired socially, not statistically, which creates both opportunities and risks when interacting with systems that can simulate human communication so convincingly.

Another point I discussed is that modern AI systems are increasingly less “designed” in the traditional sense and more “grown”. We understand the learning algorithms reasonably well, but we do not fully understand what emerges from large-scale training. Yet this is not entirely unfamiliar territory. Humans already trust many systems we do not fully understand, from the human brain itself to turbulence and aerodynamics in aviation.

Trust does not come solely from complete understanding. It comes from rigorous evaluation, understandable controls and guardrails around the less understandable parts, human and institutional accountability, and the ability to intervene when things go wrong.

One other thought struck me while sitting inside a communication museum. Communication technologies used to mainly distribute information. AI increasingly mediates interpretation itself. Information is now generated, filtered, summarised, prioritised, and explained by AI before humans even encounter it.

I have to admit, slightly guiltily, that before the panel I hadn’t yet read Grace Chan’s novel Every Version of You. Instead, I asked AI not just to summarise it, but to “pretend to be Grace Chan and tell me about the novel”.
That felt simultaneously useful, fascinating, unsettling, and slightly dystopian.

One line the AI generated particularly stayed with me: “Technology used to extend memory. Now it extends identity.”

To reduce my guilt somewhat, I did buy the book after the panel. Hopefully I can get Grace’s actual autograph someday instead of the machine-generated version