Welcome to 2019! Let’s start the year off by discussing how your ability to exercise your free will could be directly impacted by who you follow on social media, or who you pass by on the street.
In the dystopian Netflix series Black Mirror, there is an episode called Nosedive where a person’s ability to ride a plane, work at their job or even be served food at a restaurant is decided by a crowd-sourced rating system. Much like you rate your Uber driver at the end of your trip (and your Uber driver rates you), in this episode, every social interaction is followed by both parties giving each other a star-rating with an app on their phone. Everyone’s rating is public and it seems like you can downvote someone at any time. People with higher overall scores have more influence on the scores of people that they interact with. Inevitably, the episode’s main character encounters a string of bad luck and unfortunate encounters that lead to her score tumbling to almost nothing. She ends up losing her job, her home and her ability to travel in a matter of days based on nothing more than a few left swipes from some strangers and a co-worker.
Everyone has bad days, so it really hits home when you see the protagonist lose her life over a few social mishaps. She doesn’t actually die, but she is locked in a plastic box because of her low social score.
How far are we from experiencing this kind of scenario in our lives?
The Chinese government has announced plans to implement a Social Credit Score for its citizens, that tracks all online and in-person activity, including who your friends are, who you talk to and who your acquaintances talk to. This score will decide if you can get a loan on a house, similar to a credit score in the United States or Canada, but it will also decide what schools you can go to, what businesses you can shop at and even whether or not you can leave the country. Going live in 2020, literally next year, all 1.6 billion Chinese residents will be subject to this Social Credit System; a system that is eerily similar to the one described in the Black Mirror episode.
I live in North America. Calgary, Alberta, Canada, specifically, so this won’t affect me, right?
All around the world, people have free and easy access to instant global communication networks, the wealth of human knowledge at their fingertips, up-to-the-minute information from across the earth, and unlimited usage of the most remarkable software and technology, built by private companies, paid for by adverts. That was the deal that we made. Free technology in return for your data and the ability to use it to influence and profit from you. The best and worst of capitalism in one simple swap. We might decide we’re happy with that deal. And that’s perfectly fine. But if we do, it’s important to be aware of the dangers of collecting this data in the first place. We need to consider where these datasets could lead – even beyond the issues of privacy and the potential to undermine democracy (as if they weren’t bad enough). There is another twist in this dystopian tale. An application for these rich, interconnected datasets that belongs in the popular Netflix show Black Mirror, but exists in reality. It’s known as Sesame Credit, a citizen scoring system used by the Chinese government. Imagine every piece of information that a data broker might have on you collapsed down into a single score. Everything goes into it. Your credit history, your mobile phone number, your address – the usual stuff. But all your day-to-day behaviour, too. Your social media posts, the data from your ride-hailing app, even records from your online matchmaking service. The result is a single number between 350 and 950 points. Sesame Credit doesn’t disclose the details of its ‘complex’ scoring algorithm. But Li Yingyun, the company’s technology director, did share some examples of what might be inferred from its results in an interview with the Beijing-based Caixin Media. ‘Someone who plays video games for ten hours a day, for example, would be considered an idle person. Someone who frequently buys diapers would be considered as probably a parent, who on balance is more likely to have a sense of responsibility.’ If you’re Chinese, these scores matter. If your rating is over 600 points, you can take out a special credit card. Above 666 and you’ll be rewarded with a higher credit limit. Those with scores above 650 can hire a car without a deposit and use a VIP lane at Beijing airport. Anyone over 750 can apply for a fast-tracked visa to Europe. It’s all fun and games now while the scheme is voluntary. But when the citizen scoring system becomes mandatory in 2020, people with low scores stand to feel the repercussions in every aspect of their lives. The government’s own document on the system outlines examples of punishments that could be meted out to anyone deemed disobedient: ‘Restrictions on leaving the borders, restrictions on the purchase of . . . property, travelling on aircraft, on tourism and holidays or staying in star-ranked hotels.’ It also warns that in the case of ‘gravely trust breaking subjects’ it will ‘guide commercial banks . . . to limit their provision of loans, sales insurance and other such services’. Loyalty is praised. Breaking trust is punished.
Fry, Hannah. Hello World: Being Human in the Age of Algorithms (pp. 44-46). W. W. Norton & Company. Kindle Edition.
Science fiction mirrors reality and reality reflects science fiction. While this type of system is only going live in China, it is unclear how long we have before something similar arrives in the West.