It was a pleasure to deliver a talk last week at The Future Generation Enterprise Architecture (FGEA) Symposium hosted by ACS (Australian Computer Society) and organised by Asif Gill.
I focused on data trust at scale, challenging some conventional views on data quality in distributed, at-scale, AI-amenable environments. Some lessons come from CSIRO’s Data61 technology powering data.gov.au, the early years of hosting the Data Standards Body for Australia’s Consumer Data Right (e.g. open banking), and cross-organisation/border/supply chain data flow projects, not to mention the science and tech development for data/AI safety. Here are a few key points and selected slides:
๐๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐๐ญ๐ โ ๐๐๐ฒ๐จ๐ง๐ ๐๐ข๐๐ฌ ๐๐ง๐ ๐๐๐ฉ๐ซ๐๐ฌ๐๐ง๐ญ๐๐ญ๐ข๐จ๐ง
* Distributed Trust: Ensuring data integrity across various organisations or on-device data without direct oversight.
* Zero Trust: Unsupervised learning from online wild data is susceptible to data poisoning attacks and can’t be easily cleaned up.
* Trusted “License”: It’s about data rights and value redistribution, not just ownership or copyright.
* Trusted Artificial: Synthetic data can be useful, but when should we trust it?
๐๐๐ฌ๐ญ๐ข๐ง๐ /๐๐๐ฅ๐ข๐๐๐ญ๐ข๐จ๐ง ๐๐๐ญ๐ โ ๐๐๐ฒ๐จ๐ง๐ ๐๐ฎ๐ฆ๐๐ง ๐
๐๐๐๐๐๐๐ค ๐๐ฌ ๐ญ๐ก๐ ๐๐จ๐ฅ๐ ๐๐ญ๐๐ง๐๐๐ซ๐
* Trust in Evaluation Data: Accidental data leaks can invalidate testing outcomes, so tread carefully.
* Trust in Human Feedback: Often unreliable, using human feedback necessitates nuanced evaluation.
๐๐ฒ๐ฌ๐ญ๐๐ฆ-๐๐๐ฏ๐๐ฅ ๐๐๐ญ๐ – ๐๐๐ฒ๐จ๐ง๐ ๐๐จ๐๐๐ฅ ๐๐๐ญ๐
* Trusted Knowledge: Itโs not just about training data. Addressing inconsistencies within your inference-time data sources and between external knowledge and AI knowledge is crucial.
* Trusted Trade-off: Itโs never about single-dimension optimisation. Balancing privacy, fairness, and accuracy requires stakeholder involvement before and after deployment in context.
* Trusted Provenance: Ensuring data provenance throughout its lifecycle is essential to combat low-quality decision and misinformation.
Slides – see LinkedIn post: https://www.linkedin.com/feed/update/urn:li:activity:7214394551518519296/
