This article explains how Riposte turns your posts into aggregated wellbeing data analytics using a machine learning model built for diversity.
Riposte does use contributors’ data and it is important to us that people understand why, how, and for what, so that they can make informed decisions on sharing their data with us. In this post, we share how we use data and for what, using a real example from a highschool in Aotearoa New Zealand. In another blogpost about the purpose of Riposte we go into further detail about the why. This is a story about Covid-19 and wet socks.
This article is part of a series that introduces Riposte, a new social sharing and wellbeing app from Aotearoa New Zealand that uses data for collective wellbeing measurement. The other parts of the series focus on designing social media to be healthy and psychologically safe, paying people for their data, and using data for good as an alternative social media business model, and our purpose and vision.
Covid-19 and the mystery of the soggy socks
In June 2020, Aotearoa New Zealand came out of lockdown and students went back to school. Prefects from one particular highschool employed the use of the Riposte app to gauge how students were feeling about returning. The students were introduced to the app and started sharing their highs and lows each day. Each post fed into the collective dataset of the school, which ran through the bespoke wellbeing model built by Riposte. From that, our analytics team provided the prefects with wellbeing reports of aggregated data showing trending topics.
One week, our team saw a sharp drop in physical wellbeing and “wet socks” was trending as a negative topic. They started wondering what could be making student’s socks wet. Was the changing room flooded? Were students caught off guard by rampant sprinklers? Was this a new form of bullying? What was going on here? Our team presented their findings to the prefects and highlighted the mystery of the soggy socks. In response, the prefects engaged with students and together they identified the root cause.
To understand the cause of this issue, it’s important to know that some highschool students in Aotearoa New Zealand are required to wear uniforms. In summer, students wear sandals. In winter, closed shoes. In Gisborne, many students buy their school uniform shoes online, but due to the Covid-19 lockdown, supply chains were disrupted, and the winter shoes hadn’t arrived when students went back to school. Most students did have weather appropriate shoes at home, but school uniform protocol doesn’t allow them. So students wore their summer uniform sandals in rainy late autumn weather. The result: wet socks.
Another important thing to understand in this context is that many students would not speak up about their wet sock issue directly to teachers. However, it did disturb them so much that enough students posted about it on Riposte for it to be trending, where it was picked up by our analytics team and presented to back to the prefects. The prefects in their leadership roles could then actively ask about the issue and what was causing it.
And so, the mystery of the wet socks was solved. Next time the school goes into lockdown between seasons they will be aware of this challenge in advance and be able to plan accordingly. This example shows a concrete case of how Riposte analytics data was used to detect a relevant wellbeing issue of a particular group.
What happens to each individual post?
To find out what happens to an individual post, let’s follow one hypothetical post in this dataset. Say a student, let’s call her Renay, posted a facepalm (negative post) saying: “Aarg, wet socks!”, marked it as “extremely frustrated” and added “#wet cold feet #backtoschool”. When she posts this, the post is stored in a centralised database. Her post is stored on an AWS (Amazon Web Services) server in Sydney which adheres to Aotearoa New Zealand legal data security requirements.*
In the database, Renay’s post is separated from her personal details through a unique key. She is a student at the highschool Riposte is working with and has opted into the school’s Riposte group by entering a code in her profile settings, so that school is added as one of the research groups that her post contributes to.
Renay has been informed about her post being used in analytics: when she signed up to the app she was required to tick a box stating that she understands that Riposte uses her post for wellbeing analytics. At Riposte, we consciously avoid dark patterns in our design, because we want people to know what happens to their data. That’s why this box is in addition to the standard terms and agreement box and must be ticked separately. When Renay posts and chooses who can see her post, she is informed again that her post will feed into the collective dataset for wellbeing analytics.
Now the post is in the database as part of the collective dataset. It is identifiable as a post belonging to the highschool research group and is used when analysing data from the students as a collective. The raw data is only accessed by our analytics team. As posts are immediately separated from contributors’ personal data through a unique key, the analysts do not see who posted what when accessing the data.
Once in the database, the post is scanned by three machine learning algorithms. The first is programmed to detect toxicity. This model scans for any swear words or otherwise rude language. If the algorithm finds a positive match, it marks the post as pending approval and a human in our team has to read and approve or disapprove the post. If a post is disapproved, it is not visible to others in the app but still feeds into the collective dataset. The second algorithm is programmed to detect mental distress. If a positive match is found here, an email is sent to the contributor with information on how to find extra support if it is needed. After going through the toxicity and mental distress algorithms, the post is analysed with the bespoke Riposte wellbeing model that categorises each post as falling under one, two, or three wellbeing categories (for example mental & emotional, physical, spiritual, cultural, environmental, social wellbeing etc).
What happens with the aggregated data?
Once a post has become part of the collective dataset and that dataset has gone through the Riposte wellbeing model, the analytics team can use the aggregated data to report on wellbeing trends of that group of contributors. In the ‘wet socks’ example, this meant that Renay’s post contributed to the trend of a drop in physical wellbeing recognised by the model accompanied by frequent mention of wet socks. This trend was highlighted in feedback to the prefects of Renay’s school.
Another example of concrete action in response to trends in the students’ collective wellbeing happened later that year leading up to exam time. Multiple schools were engaged in a campaign that we ran for with a youth employability programme called Licence to Work. The data showed increased levels of anxiety related to time-pressure leading up to exam time, students struggling to sleep, and not coping well with the workloads. Our campaign team had originally planned to give out clothing vouchers as spot prizes during this time, but when they saw such a strong trend of exam-anxiety, they quickly pivoted their plan and instead put together “Exam Survival Packs” as spot prizes. These prizes included sleep drops, relaxing teas, calming body washes, and other stress-reducing products. Students’ loved these, saying they were much more relevant than clothing.
The reports made by Riposte show net wellbeing scores (positivity levels), an analysis of trending topics, and top keywords used by contributors. Our analytics team can also provide before and after, or periodic comparisons, or give access to an interactive dashboard for real time insights. This enables insight into large groups of people, while letting them speak from their own experience and in their own words. In this aspect, Riposte differs radically from surveys which are generally used to gather wellbeing data. Surveys have a lot of pre-defined language, and youth in particular express that they don’t like filling them out. The data gathered through the Riposte app is emergent. It shows which topics are relevant to people, literally in their own terms. This can prompt those who make decisions that affect people’s wellbeing to ask new questions, gain a different understanding of possible root causes to problems, and come up with tailor-made solutions to address those problems.
A note on diversity and representation in building our wellbeing model.
The bespoke Riposte wellbeing machine learning model was purposely built by a diverse team. The model is an emergent data science model that evolved over a period of six months, using over 4000 posts from a wide range of contributors. The contributors included highschool students, city council workers, orchard workers, co-workers from a startup hub, and primary school teachers. As the model was built, the team building it taught it how to relate words in posts to themes and how to categorise those themes. The team consisted of six people: a data scientist, a data analyst, a sociology student, a web developer and sociology student, an IT administrator and forum moderator, and a quality control and health and safety specialist. Of these six people, there were four females, three Māori, two Pākeha (white New Zealanders), and one Indian. Their ages ranged from in their 20’s to 60’s with one person in each decade. The diversity of the team was deliberate, because we are aware that data violence is a real problem. Building a wellbeing model requires a lot of small decisions around categorisation and language, and those decisions will influence people using the product. Therefore, at Riposte we decided that one thing we could do, was ensure the model was made by a team that is as representative as possible of the diversity of the wider population.
Grounding innovative approaches in emergent data.
In Aotearoa New Zealand, societal challenges such as mental health, housing, poverty, and physical health are persistent. These challenges may require creative methods and experimentation to find ways of addressing them that can be effective in our rapidly changing world. It’s important that new solutions are still grounded in data.
Riposte data provides timely and ongoing insights into the wellbeing of New Zealanders. As the number of contributors grows both locally and internationally, this data can become a valuable tool for decision makers that are advising on, deciding on, and implementing policies, programs, and even highschool prizes.
We look forward to a future in which all decisions take into account the wellbeing of people they affect. We think this is a part of the transition towards a future that prioritises people and planet over profit, so we are doing our bit to move in that direction. Are you interested in that future and keen to connect with us? Get in touch or come by for a coffee.
*This adheres to New Zealand regulations, but is not in line with the principles of Māori data sovereignty as expressed by Te Mana Raraunga. The principles state that “whenever possible, Māori data shall be stored in Aotearoa New Zealand”. Riposte is conscious of this and actively looking for a solution that enables this whilst retaining the level of security required. Any practical advice on this is most welcome.
About the author: Mariska van Gaalen is a Dutch-Kiwi who moved back to Aotearoa New Zealand after 26 years abroad. Her background is in sustainable development, mostly as a researcher and writer. She has settled in Tairāwhiti and is keen to work more locally supporting positive impact projects and ventures. She currently writes for Riposte, is learning how to grow oyster mushrooms zero waste, and does the occasional stint sanding houses. Mariska is also part of the crew at Tāiki E! Impact House, where Riposte is based.