I often share missed or surprising observations in my talks. Here are five of them, summarised as food for thought.

𝗠𝘆𝘁𝗵 𝟭: 𝗢𝘃𝗲𝗿𝘀𝗲𝗮𝘀-𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗔𝗜 𝘄𝗼𝗻’𝘁 𝗮𝗹𝗶𝗴𝗻 𝘄𝗶𝘁𝗵 𝗔𝘂𝘀𝘁𝗿𝗮𝗹𝗶𝗮𝗻 𝘃𝗮𝗹𝘂𝗲𝘀.
It’s intuitive to think globally trained models can’t reflect our norms. Yet when frontier models answer the same cultural questions posed to national cohorts, they align most strongly with Australia and New Zealand—above the US [1]. There are many possible reasons, but 𝘵𝘩𝘦 𝘪𝘯𝘴𝘪𝘨𝘩𝘵 𝘪𝘴 𝘤𝘭𝘦𝘢𝘳: 𝘸𝘦 𝘯𝘦𝘦𝘥 𝘳𝘪𝘨𝘰𝘳𝘰𝘶𝘴 𝘦𝘷𝘢𝘭𝘶𝘢𝘵𝘪𝘰𝘯, 𝘯𝘰𝘵 𝘫𝘶𝘴𝘵 𝘪𝘯𝘵𝘶𝘪𝘵𝘪𝘰𝘯.

𝗠𝘆𝘁𝗵 𝟮: 𝗧𝗼 𝘀𝗼𝗹𝘃𝗲 𝗮 𝗽𝗿𝗼𝗯𝗹𝗲𝗺, 𝘆𝗼𝘂 𝗺𝘂𝘀𝘁 𝘁𝗿𝗮𝗶𝗻 𝗼𝗻 𝗱𝗮𝘁𝗮 𝘁𝗵𝗮𝘁 𝗿𝗲𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝘀 𝗶𝘁.
We’re told models need domain-matched data in training. Yet the strongest systems for essays/poetry improve when trained with software code, data unrelated to the task domain. Some data also plays an outsized role only during inference and has little effect during training. 𝘛𝘩𝘦 𝘭𝘦𝘴𝘴𝘰𝘯 𝘪𝘴 𝘤𝘰𝘯𝘴𝘪𝘴𝘵𝘦𝘯𝘵: 𝘦𝘷𝘢𝘭𝘶𝘢𝘵𝘦 𝘸𝘩𝘦𝘳𝘦 𝘶𝘯𝘪𝘲𝘶𝘦 𝘥𝘢𝘵𝘢 𝘩𝘢𝘴 𝘳𝘦𝘢𝘭 𝘪𝘮𝘱𝘢𝘤𝘵, 𝘯𝘰𝘵 𝘫𝘶𝘴𝘵 𝘸𝘩𝘦𝘳𝘦 𝘪𝘵 𝘭𝘰𝘰𝘬𝘴 𝘳𝘦𝘭𝘦𝘷𝘢𝘯𝘵.

𝗠𝘆𝘁𝗵 𝟯: 𝗔𝗜 𝗶𝘀 𝗮 𝗴𝗲𝗻𝗲𝗿𝗮𝗹-𝗽𝘂𝗿𝗽𝗼𝘀𝗲 𝘁𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆 𝗹𝗶𝗸𝗲 𝗲𝗹𝗲𝗰𝘁𝗿𝗶𝗰𝗶𝘁𝘆 𝗼𝗿 𝗰𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴.
Electricity and computing required further human invention to build each application. Today’s AI began as “predict the next word”, yet a bundle of narrow capabilities automatically emerges without human design. In practice, it behaves less like a general platform and 𝘮𝘰𝘳𝘦 𝘭𝘪𝘬𝘦 𝘢 𝘭𝘪𝘣𝘳𝘢𝘳𝘺 𝘰𝘧 𝘳𝘦𝘢𝘥𝘺-𝘮𝘢𝘥𝘦 𝘧𝘶𝘯𝘤𝘵𝘪𝘰𝘯𝘴 𝘵𝘩𝘢𝘵 𝘴𝘵𝘪𝘭𝘭 𝘳𝘦𝘲𝘶𝘪𝘳𝘦 𝘴𝘮𝘢𝘳𝘵 𝘦𝘭𝘪𝘤𝘪𝘵𝘢𝘵𝘪𝘰𝘯, 𝘳𝘪𝘨𝘰𝘳𝘰𝘶𝘴 𝘦𝘷𝘢𝘭𝘶𝘢𝘵𝘪𝘰𝘯, 𝘢𝘯𝘥 𝘴𝘢𝘧𝘦𝘵𝘺 𝘤𝘰𝘯𝘵𝘳𝘰𝘭 𝘰𝘶𝘵𝘴𝘪𝘥𝘦 𝘵𝘩𝘦 𝘮𝘰𝘥𝘦𝘭.

𝗠𝘆𝘁𝗵 𝟰: 𝗦𝗮𝗳𝗲𝘁𝘆 𝗮𝗻𝗱 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻 𝗮𝗿𝗲 𝗮 𝘁𝗿𝗮𝗱𝗲-𝗼𝗳𝗳.
The assumption that you must choose between safety and performance is outdated. Training for robustness against prompt attacks, for example, tends to improve reasoning and generalisation on challenging benchmarks, turning guardrails into a performance gain. And roughly 70% of safety improvements also arrive alongside general capability gains. 𝘛𝘩𝘦 𝘵𝘸𝘰 𝘳𝘦𝘪𝘯𝘧𝘰𝘳𝘤𝘦 𝘦𝘢𝘤𝘩 𝘰𝘵𝘩𝘦𝘳 𝘳𝘢𝘵𝘩𝘦𝘳 𝘵𝘩𝘢𝘯 𝘤𝘰𝘮𝘱𝘦𝘵𝘦.

𝗠𝘆𝘁𝗵 𝟱: 𝗛𝘂𝗺𝗮𝗻 𝗼𝘃𝗲𝗿𝘀𝗶𝗴𝗵𝘁 𝗻𝗮𝘁𝘂𝗿𝗮𝗹𝗹𝘆 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝘀 𝗼𝘂𝘁𝗰𝗼𝗺𝗲𝘀.
Adding humans into the loop sounds safe and performance-enhancing, yet naïve oversight often underperforms both AI and human alone. Without the right tools, context, and incentives, reviewers can either over/under-trust or disengage. 𝘖𝘷𝘦𝘳𝘴𝘪𝘨𝘩𝘵 𝘰𝘯𝘭𝘺 𝘸𝘰𝘳𝘬𝘴 𝘸𝘩𝘦𝘯 𝘪𝘵’𝘴 𝘱𝘶𝘳𝘱𝘰𝘴𝘦-𝘥𝘦𝘴𝘪𝘨𝘯𝘦𝘥 𝘧𝘰𝘳 𝘶𝘯𝘥𝘦𝘳𝘴𝘵𝘢𝘯𝘥𝘪𝘯𝘨 𝘢𝘯𝘥 𝘭𝘦𝘢𝘳𝘯𝘪𝘯𝘨.

At CSIRO’s Data61, we’re operationalising these insights. Talk to us if you are interested.

[1] https://www.adalovelaceinstitute.org/blog/cultural-misalignment-llms/ See Figure 1: the higher a country appears on the vertical axis, the more closely the AI answers align with the answers given by people from that country