It is not everyday that you are asked and allowed to behave badly in the name of science. 5 participants of the workshop ‘Unethically me’ at NordiCHI’20 were asked to do so on the 26th of October. The participants were asked to behave badly in order to investigate how Artificial Intelligence’s explanation could be used unethically and more importantly to discover ways to counter such behaviour. During the 5 hour workshop we explored a reality where 4 unethical personas were tasked to abuse the explanations of an algorithm for their purposes. This article describes the experience.
On the 26th of October a group of researchers from imec-SMIT – VUB were invited to organise a workshop for the NordiChi’20 conference. This conference is focused on computer-human-interaction from the perspective of the Nordic countries and is held every 2 years. This year the conference was supposed to take place in Tallinn, due to Covid-19 it became an online workshop instead of an in-person affair. Armed with a digital workshop board and a zoom-callroom we went into the meeting.
After the success and failure to have people draw a car and a cup of coffee while only providing instructions in the form of shapes. We discussed what we understood with “explainability”. We agreed that we all understood it to mean the ability of a Machine learning model to explain itself to the general public and we agreed with Bhat et all (2020):
“Explainability” loosely refers to any technique that helps the user or developer of ML models understand why models behave the way they do.”(Bhatt et al. 2020)
We then asked the participants to create their own versions of unethical personas. They had received the bare descriptions of the main objectives in order to make it possible to personalise the persona.
- Jordan: only objective is to make money
- Riley: wants to disadvantage only certain groups of people
- Sam: chaos is the goal
- MacKenzie: wants sustainability and this is the only concern of this persona
At first glance MacKenzie seems to be the outlier as the goal is an ethical one. However, it became clear that by solely focusing on sustainability and ignoring other ethical concerns this persona could be encouraged to make unethical explanations.
The participants imagined themselves as these persona’s and were asked to engage with two hypothetical ML models and to provide explanations in the guise of their persona. And to examine how and why their persona acted in such a way. And finally to define what made these actions unethical. An example of this is persona Sam who decided that in order to sow chaos they would have their shopping recommendation system only provide information about non-existent promotions or environmental unfriendly products.
After creating the unethical explanations and behaviour, the workshop focused on how we could counter this and we created a top 8 list of steps that we should be able to make in order to thwart unethical intentions:
- Prioritise ethical thinking within development
- Ensure ethics education for your employees
- Identify who the users are
- Involve users in the testing phase
- Involve the target group early on (not just in the end)
- Have a close look at the data (training data for biases)
- Have a dedicated team for ethical consideration of the project
- Changes over time can happen: keep an eye on changes in ethical perspectives
We closed off on the hopeful note that being unethical could be fun but thinking about countering the behaviour was more satisfying in the long run.
(Photo by Duncan Shaffer on Unsplash)