Measurementality
Tweet #measurementalityAt the end of each month we’ll be hosting a Webinar featuring the same guests who were on the Radical AI Measuremenality podcast. We’ll also invite additional guests who are working in IEEE SA committees, Standards Working Groups, or other programs. That way we can further explore the themes and issues brought up on the podcasts to best grow our work with relevance and specificity.
Latest Webinar
Webinar #7: Measurementality: Why AI Registries are Critical for Metrics of Accountability
Date: 26 August @ 12:00 PM ET - 1:00 PM ET
To know the consequences of an organization’s use of Artificial Intelligence Systems (AIS), it is essential that they know what AIS models they are deploying and where. At present, not all AIS companies using AIS can point to each of their models, metrics, data sources, personnel controls, and deployment documentation. To get to a state where all AIS could be accounted for and all of the channels through which the power of AIS flows, companies and governments should explore the concept of an AIS or AI registry. In this episode of Measurementality, we discuss registries, including their content, purpose, and the personnel who are best positioned to create and maintain these for their organizations. Knowing more about AIS / AI registries will help organizations improve their AIS governance, integrate AIS governance with an overall corporate sustainability and strategy effort, and prepare for coming regulatory changes in the AIS space. As a form of Responsible Innovation, metrics regarding these registries may or should become the new norm in AI Governance.
Join Measurementality host John C. Havens as he interviews Dr. Sara R. Jordan, Senior Researcher, Artificial Intelligence and Ethics at the Future of Privacy Forum and Ilana Golbin, Director of Emerging Technologies & AI and Global Responsible AI Leader at PwC.
View Webinar
We’re delighted to be working with the hosts and creators of The Radical AI Podcast to collaborate on and sponsor this series.
The first ten episodes of The Measurementality podcast will be hosted on Jess and Dylan’s site as a way to help cross pollinate awareness between the Radical AI and IEEE SA communities. Episodes will begin in January, 2021 and will be posted on the Radical AI site in the beginning of each month.
iTunes | Spotify | iHeartRadio
Latest Episodes
Episode #2: Measurementality: Children's Data and Sustainability
Date: 22 March 2021
Jess and Dylan from The Radical AI podcast interview Sandy Pentland of MIT and Baroness Beeban Kidron of the 5Rights Foundation in their latest installment of Measurementality. Focusing on the key goal of our series, "defining what counts in the algorithmic age," Sandy and Beeban discuss issues including data privacy for children, data agency for all, and how metrics like the United Nations Sustainable Development Goals and other human rights oriented metrics are being utilized in the design of Artificial Intelligence Systems (AIS).
Podcast #3: Measurementality: Counting Mental Health and Caregiving
Join Jess and Dylan from The Radical AI podcast as the Measurementality series continues to define "what counts in the algorithmic age" with guests including Dr. Riane Eisler, president of the Center for Partnership Studies (CPS) and author of The Real Wealth of Nations: Creating a Caring Economics, hailed by Archbishop Desmond Tutu as “a template for the better world we have been so urgently seeking” and Amandeep Gill, Director of the Global Health Centre project on International Digital Health & AI Research Collaborative (I-DAIR), and former Executive Director and co-Lead of the Secretariat of the UN Secretary General’s High-Level Panel on Digital Cooperation. Questions discussed on the show include, What metrics do you trust that guide your efforts to improve your mental health; How do you value the caregivers in your lives; and, how does the fact that caregiving metrics aren't a part of GDP/ economic Indicators affect how society and AI Systems design prioritize mental health and caregivers - or not?
Podcast #4: Measurementality: What are We Optimizing For?
In this episode we'll be "identifying what counts in the algorithmic age" by analyzing how existing metrics regarding human wellbeing along with environmental flourishing are being globally measured today.
While it's often assumed that human "wellbeing" refers to moods or emotions that can't be measured, the fields of positive psychology and happiness indicators provide multiple objective and subjective ways to measure long term human flourishing that can and should be utilized in the design of AI. Likewise, environmental metrics that often are used solely as ways to minimize harms after a product or service are built are not keeping our planet from deterioration faster than we can address it via normal means of Corporate Social Responsibility (CSR) or Environmental, Social, and Governance (ESG) reporting. So how can we better understand what we're optimizing for in the AI we create to include people and planet in design measures at the outset of everything we create?
Join Jess and Dylan, hosts of the Radical AI podcast as they interview Laura Musikanski, Executive Director of The Happiness Alliance and Chair of IEEE 7010-2020; and Jonathan Stray, former research partner at The Partnership on AI and author of Aligning AI to Human Values means Picking the Right Metrics to learn more.
The question of defining what counts needs as much feedback as we can receive from as many people in as many different regions of the world from as many backgrounds as possible.
We’ll be creating a first draft of our report after Episode Six to be released in the summer of 2021, and a second / final draft to be released at the end of the year. Both reports will feature answers from our podcasts, webinars, and your responses.
The Measurementality program poses the following three questions:
- How is success measured today in AIS? What issues (positive and negative) do you see with these measures with an eye towards human welfare, environmental sustainability, responsibility, technology, economics and ethics?
- What is the positive future you’re working to build with AIS? Answers here can be submitted in short story form as well as prose.
- What are the measures of success for that positive future? Please be as specific as possible, utilizing sample Indicators or criteria.
But we would also like to hear from you!
We will be posting all submissions received in two public documents available from the IEEE SA website featuring Measurementality at approximately the same time as our two Measurementality reports are published. We will email all people who provided submissions before the reports or documents featuring feedback are published. Submission Guidelines (PDF).
For the co-hosts of The Radical AI Podcast, Dylan and Jess, two PhD students rooted in values of curiosity, humility, and hope for a better world, they began to interview dozens of scholars in the responsible tech community about a concept they coined “radical AI” and what it means to them, and what unconventional work might look like in AI. Their goal with this project became to resist harmful technologies and how they are created today through community building and collective storytelling, and to continue to co-define this concept. Their Radical AI Podcast and community is still in active exploration of what this work really is in the AI field.
For IEEE SA, we serve as a convener around critical ideas related to technology. When The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems gathered more than seven hundred global thinkers to create Ethically Aligned Design, many of the ideas that stemmed from that work may have seemed unconventional or “radical.” Prioritizing values-based design in systems engineering, for instance. But the multi-disciplinary process of bringing volunteers together from the US, EU, UK, China, Japan, Korea, Australia, India and many other countries meant that one person’s definition of one word or concept meant something completely different to someone else.
So while it may seem simple, actually listening to other people with consensus and curiosity can be one of the most unconventional things anyone can do.
So with the help of Jess and Dylan, our global experts and interviewees, and all of you, that’s exactly what we’re going to do. Start listening to your answer to a the question:
“What counts most in your life?"