Back from New Delhi, and after letting the conversations and ideas settle for a few days, here are my final reflections from the AI Impact Summit.

As a scientist, the most valuable moments were not the plenaries, but the deep, sometimes once-in-a-lifetime discussions with individual AI pioneers building AI in frontier labs, startups and research organisations. Comparing notes on what is actually working, what is not, and what is coming next was far more revealing than any formal aspiration.

A few scientific observations, some slightly contrarian, stood out.

First, using AI is not about inserting it into existing workflows. It is about recasting the problem so it becomes easier for AI to solve. The breakthrough pattern is not human checking of outputs, but enabling cheap, automated verification. If you can exploit solve–verify asymmetry, wrong answers become part of structured exploration. With automated feedback, AI can search towards correctness rather than wait for humans to supervise every step. This validates the verification-first approach [1] and the idea of human oversight [2] at the meta-level, steering and evaluative layer, rather than at the operative layer or instance-by-instance human verification.

Second, the most potent learning for an organisation happens post-training. The decisive factor is often not which models you have chosen or deployed after careful development and testing, but whether you build tight feedback loops, verification layers and smart context so that daily use enables further learning. Your system learning architecture and context engineering shape the effective intelligence of your AI system. Fun fact: we actually coined the term “context engineering” [3] in late 2023 and early 2024, well before it became fashionable in mid-2025. As often happens, research runs ahead of practice by years.

Third, AI’s strength is not only extracting patterns from domain-specific data, but forming connections across domains. Some data types, such as software code, seem to enhance general capability across many domains. Other highly specialised datasets can degrade general intelligence. That may partly explain why, despite the hype, there are still relatively few real-world deployments of fine-tuning, which is different from the wider set of post-training techniques. As Sir Demis noted, what belongs inside AI intelligence and what should remain external as tools remains an empirical question. I argued in my ICSE 2023 keynote [4] that deciding and managing which external capabilities/tools may eventually be absorbed into AI is a key design choice. Yet because what is inside an AI model is not fully interpretable, we may deliberately keep certain controls and guardrails outside, even if it is not the most efficient way, simply to preserve clarity of human intention, agency and controllability .

If there is one slightly contrarian takeaway from the week, it is this: advantage will belong less to those with access to bigger or more aligned models, or clever prompts, and more to those who can reshape problems, engineer context and post-training learning, build system-level guardrails and automated verification at scale, and upskill the capabilities required to do that.

At CSIRO’s Data61, we are operationalising these insights as we speak. Talk to us if you are interested.

[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6031534

[2] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5501939

[3] https://ieeexplore.ieee.org/document/10553223

[4] https://liming-zhu.org/icse-2023-keynote-software-engineering-as-the-linchpin-of-responsible-ai/