The worst of surveillance capitalism
I had never written about Facebook before, and yet this company represents the worst of advertising surveillance. The consequences of its domination are serious:
- Addiction: Facebook's product teams have one ultimate goal: growing engagement. The more time you spend on its apps (Facebook, Instagram, Messenger, WhatsApp), the better you will be “monetized”.
- Radicalization: extreme content provokes reactions and drives engagement on the platform. As a result, Facebook's algorithms promote extreme content and misinformation. Facebook is a boon for conspiracy theorists, fanatics and the far right, endangering democracy in many countries.
- Censorship: Facebook and Instagram are almost unavoidable for anyone who wants to inform or alert. But moderation rules are arbitrary and appeals are complicated. In some countries there are not enough moderators, and they do not understand cultural subtleties well. Many members of civil society and activists find themselves censored.
- Trauma for moderators: they are contractors, poorly regarded and poorly paid, confronted with horrors every day. They often remain traumatized for a long time.
- A company worthy of 1984: with its partner Google, Facebook played a foundational role in the installation of generalized surveillance in our societies, undermining democracy.
The very provocative but honest memo 'The Ugly', written in 2016 by one of Facebook's executives, sums up the company culture well: “Growth at any cost”. Here is an excerpt:
So we connect more people That can be bad if they make it negative. Maybe it costs a life by exposing someone to bullies. Maybe someone dies in a terrorist attack coordinated on our tools. And still we connect people.
The ugly truth is that we believe in connecting people so deeply that anything that allows us to connect more people more often is de facto good. It is perhaps the only area where the metrics do tell the true story as far as we are concerned.
And at the heart of Facebook's model is the abusive exploitation of your personal data. Facebook's disregard for its users' privacy is now widely documented on the web. But it is rare to find a story that traces the erosion of your privacy by Facebook while putting it in parallel with the state of the competition.
Dina Srinivasan is a researcher working on these subjects, at the intersection of "antitrust" and "privacy"; I had already mentioned her work in the article "The dominance of Google's advertising markets". Here, I will start from her paper "The Antitrust Case Against Facebook" to describe how Facebook was able to impose its advertising surveillance on the web and apps, despite users' strong preference for privacy.
Originally, respect for privacy was a strength of Facebook
It is hard to remember today, but the social media market was initially very competitive. In 2006, the most widely used social network was MySpace. But Facebook also faced many other social networks such as Bebo, Hi5, Friendster and Orkut (owned by Google). How do you stand out in a competitive market where the product is “free”? Through quality, and the level of privacy protection quickly became an important point of differentiation.
In 2006, then, MySpace was the leader. But it was heavily criticized in the media, accused of encouraging sexual harassment, suicides or murders (some articles from the time here, there and there). The reason? Communication on MySpace was too open, with little consideration for users' privacy.
Facebook therefore had an avenue to differentiate itself, which it did:
- MySpace was open to everyone, Facebook was initially reserved for students, who could provide a university email address (in ".edu").
- By default, MySpace user profiles were open to everyone. In the early days of Facebook, only friends and students from the same university could view each other's profiles.
- Facebook quickly gave users a lot of control, which MySpace did not: the choice to open or close one's profile to friends, friends of friends, or students at the same university. But also the ability to be visible or not in its search engine, as well as granular controls over contact information such as phone number.
Also, Facebook very quickly hired a privacy manager. Its privacy policy was short and very clear, with only 950 words. In particular, we can read:
Use of Cookies
A cookie is a piece of data stored on the user's computer tied to information about the user. We use session ID cookies to confirm that users are logged in. These cookies terminate once the users close the browser. We do not and will not use cookies to collect private information from any user.
The private network logic, the control given to users and the short privacy policy were differentiating elements compared with other social networks such as MySpace. Even if other factors may have played a role (solid technical foundation, initial elitism, cleaner user interface, etc.), stronger respect for privacy played a central role in Facebook's development.
Beacon, the first (failed) attempt at web surveillance
In 2007, Facebook became the trendy new social network (and I opened my account). In November, it launched Beacon, a transparent advertising surveillance initiative outside Facebook. At launch, The New York Times was one of the partners. If I read an article from the New York Times, Facebook would then offer, via a pop-up, to share my reading with my friends. Facebook Beacon also made it possible to share purchases, music listened to, films watched, etc.:
Facebook Beacon is watching you, but you are informed.
The presence of these new Facebook trackers allowed it to track user behavior on third-party sites (via a cookie), even when users declined to share their activity. Faced with the outcry caused by Beacon, Facebook nevertheless denied tracking users when they refused to share. Here is an excerpt from the interview with the then Vice President of Marketing & Operations at the New York Times:
Q. If I buy tickets on Fandango, and decline to publish the purchase to my friends on Facebook, does Facebook still receive the information about my purchase?
A. “Absolutely not. One of the things we are still trying to do is dispel a lot of misinformation that is being propagated unnecessarily.”
That statement was obviously contradicted a few hours later by a researcher. With Beacon trackers, Facebook also tracked users who had logged out, as well as people who did not have a Facebook account. This was Facebook's first privacy violation, contradicting its privacy policy, which at the time said only that it used cookies "to confirm that users are logged in" and that "these cookies terminate once the users close the browser".
Facebook quickly faced numerous protests, petitions and even lawsuits. Several participants in the Beacon program decided to withdraw. Other social networks also took advantage of this scandal to criticize Facebook and improve privacy management on their platforms. In early December 2007, Mark Zuckerberg apologized (for a history of Zuckerberg's apologies, here is a good article) and announced an opt-out option. The option, hidden in the settings, still did not meet users' requirements, and Facebook shut down Beacon less than a year later.
This rapid retreat is proof of a market that was still competitive. Facebook was under pressure from competing social networks. Also, in order to restore an already betrayed trust, Facebook announced in 2009 that any change to its privacy policy would now be put to a vote.
The Like button, an ideal Trojan horse
Facebook is consistent in its ideas: it learns from its mistakes. In April 2010, it introduced the Like button during its annual developer conference. For publishers, this was an opportunity to benefit from easy distribution of their articles on Facebook, and therefore attract new readers. It quickly became a success: in the first few weeks, more than 50,000 sites installed the Like button, including famous publishers such as CNN, The New York Times, The Wall Street Journal and Slate.
But as with Beacon, the Like button communicates with Facebook's servers in order to appear on your screen. Facebook can therefore track your browsing, again contradicting its privacy policy. CNET quotes the FAQ from the time:
No data is shared about you when you see a social plug-in on an external website.
Unlike Beacon, Facebook indicated that this product would not be used for commercial surveillance purposes. Two reasons:
- Facebook still remembers the Beacon scandal, and wants to avoid a new scandal.
- It has to convince publishers, competitors in the advertising market, to install these Like buttons. In theory, it could then sell The Wall Street Journal's audience to advertisers more cheaply, directly on Facebook.
As early as November 2010, a researcher detailed the leak of personal data via the Like button (Facebook Tracks and Traces Everyone: Like This!). Through cookies it installs even if you do not click on the Like button, the social network retrieves your identity, the URL of the page viewed, and the title of the article or the name of the product. Here too, surveillance takes place even if you do not have a Facebook account. But unlike Beacon, there is no Facebook pop-up asking you to share the article you are reading or the purchase you have just made: surveillance is now invisible.
How did Facebook react to these new revelations? The technical director at the time stated that these cookies were not used by Facebook to track users, but to protect user accounts from cyberattacks. As for tracking users who do not have an account, that was supposedly a bug that had since been fixed (false). The Wall Street Journal reported in its May 2011 investigation that Like buttons could track you on more than a third of the 1,000 most visited websites in the world, and on more than a million websites. Faced with such figures, we begin to grasp the scale of generalized surveillance.
In September 2011, Facebook was accused of continuing to track users even after they logged out. Facebook should delete cookies when a user logs out, in particular the user ID; it does not, and tries to deceive its audience:
Facebook does not track users across the web. Instead, we use cookies on social plugins to personalize content (e.g. show you what your friends liked), to help maintain and improve what we do (e.g. measure click-through rate), or for safety and security (e.g. keeping underage kids from trying to sign up with a different age). No information we receive when you see a social plugin is used to target ads, we delete or anonymize this information within 90 days, and we never sell your information.
Specific to logged-out cookies, they are used for safety and protection, including identifying spammers and phishers, detecting when somebody unauthorized is trying to access your account, helping you get back into your account if you get hacked, disabling registration for underage people who try to re-register with a different birth date, powering account security features such as second factor login approvals and notification, and identifying shared computers to discourage the use of “Keep me logged in.”
If we think about the best way for Facebook to impose its surveillance capitalism, it first has to convince publishers to install its trackers. Since transparency did not work (Beacon was presented as a commercial tool), Facebook preferred to operate in disguise (claiming that the Like button would not be used for advertising surveillance).
In December 2012, the Wall Street Journal returned to its investigation and reported that Like buttons were now found on two-thirds of the top 1,000 sites. Facebook again responded that it only used information from Like buttons for security and bug fixing. But the dishonesty was already there. In February 2011, Facebook had filed a patent to track users outside its own site and offer targeted advertising based on these profiles:
The present disclosure relates generally to social network systems and other websites in which users can form connections with other users, and in particular, to tracking activities of users of social network systems on other domains to, for example, analyze, target, or gauge the effectiveness of advertisements (ads) rendered in conjunction with social network systems.
Before using data collected from Like buttons for advertising surveillance, Facebook still had to clear one last hurdle: any change to the privacy policy had to be put to a vote. Here is how Facebook did it:
- In late 2012, with over a billion users and a successful IPO, Facebook proposed a vote on removing future referendums regarding its privacy policy.
- 88% of users voted against, but Facebook argued that only 589,000 people had voted (a clause required at least 30% of users to vote), so it could ignore the will of participants and abolish referendums.
Many users later complained that they had not been informed of this vote, having received no notification or email. Facebook clearly didn't want you to vote.
In June 2014, Facebook decided to activate advertising surveillance based on Like buttons (and all the other plugins it provides to publishers: Facebook Login, the Facebook pixel, the SDK for apps, etc.). The title of the article announcing the update is a model of Newspeak: Making Ads Better and Giving People More Control Over the Ads They See.
After spending 7 years promising not to exploit the data collected by its plugins for advertising surveillance, Facebook reneged on its promises. Why? The competition had been crushed (MySpace and Orkut had closed, Google+ had proved a failure) and even if the quality of Facebook was greatly degraded by this massive invasion of privacy, users no longer had viable alternatives. Facebook could now over-exploit your personal data for its own benefit.
In 2018, even the co-founder of WhatsApp advised deleting Facebook.
With WhatsApp, Facebook reproduces the same strategy
WhatsApp was created in 2009 and, very quickly, privacy protection became a major focus of the app:
So first of all, let's set the record straight. We have not, we do not and we will not ever sell your personal information to anyone. Period. End of story. Hopefully this clears things up.
In 2012, when WhatsApp co-founder Jan Koum wrote the post "Why we don't sell advertising", the stance on advertising was clear:
Advertising is not just an aesthetic nuisance, an insult to your intelligence and an interruption to your train of thoughts. At every advertising company, a significant portion of their engineering team spends their days refining data mining, improving code to collect all your personal data, improve the servers that hold all the data and make sure everything is recorded, gathered, sliced, packaged and sent... In the end, the result of all this is a slightly different advertising banner in your browser or on your mobile phone screen.
Here is another passage from the same post, dealing with the collection of personal data:
At WhatsApp, our engineers spend all their time fixing bugs, adding new features, and combing through everything to deliver rich, affordable, and reliable messaging for every phone around the world. This is our product and our passion. Your data doesn't even come into play. We simply don't care.
At the time, WhatsApp had a very good reputation. The app was appreciated for its simplicity and reliability, but also for its stance on advertising and personal data. As with Facebook in its early days, privacy protection was a competitive advantage for WhatsApp, allowing it to gain market share (against Facebook Messenger or Google Hangouts, for example).
But then, in 2014 WhatsApp sold its app to Facebook for 22 billion dollars. Given Facebook's track record, we could already fear the worst. Jan Koum nevertheless wrote on the WhatsApp blog:
Here's what will change for you, our users: nothing.
WhatsApp will remain autonomous and operate independently. You can continue to enjoy the service at lower costs. You can continue to use WhatsApp anywhere around the world or on any smartphone you use. And you can still count on absolutely no advertising interrupting your communication. There would have been no partnership between our two companies if we had had to compromise on the fundamental principles that define our company, our vision and our product.
And indeed, WhatsApp went back on its word and decided to share your personal data with Facebook starting in 2016:
But by coordinating more with Facebook, we'll be able to do things like track basic metrics about how often people use our services and improve the fight against spam on WhatsApp. And by connecting your phone number with Facebook systems, the latter can offer you better friend suggestions and show you more relevant ads if you have a Facebook account. For example, you might see an ad from a company you've worked with before instead of seeing one from a company you've never heard of.
To better understand the scope of this change, we need to step away from WhatsApp's communication. The EFF details the data now shared with Facebook: phone number, address book and usage data (when you use WhatsApp, who you communicate with, on which device, your IP address, etc.). This metadata is extremely valuable to Facebook, which, even though it does not have access to the content of your WhatsApp conversations (end-to-end encrypted), collects the most important information.
At the time, WhatsApp gave existing users only a 30-day opt-out (via this “Dark Pattern”). New users would not have this choice. And even then, the opt-out did not stop the sharing of information; it only prevented Facebook from using your WhatsApp information for targeted advertising or product improvement (friend suggestions). Let's reread the EFF on this:
Note that your WhatsApp information will still be passed to Facebook for other purposes such as “improving infrastructure and delivery systems, understanding how [Facebook and WhatsApp] services...are used, securing systems, and fighting spam, abuse, or infringement activities." Changing your settings does ensure, however, that Facebook will not use your WhatsApp data to suggest friends or serve ads.
In Europe, this data sharing went down badly. Germany refused the data sharing, then European data protection authorities “urgently” requested that personal data sharing stop. The UK data protection authority then requested that data synchronization be paused. Finally, the CNIL formally ordered WhatsApp to stop transmitting WhatsApp business intelligence data to Facebook. Meanwhile, the European Commission said Facebook had provided misleading information when buying WhatsApp and fined it 110 million euros:
Facebook told the Commission that it would not be able to automatically and reliably link the user accounts of the two companies
These episodes are very well summarized in Killian Kemps' blog post, "Does WhatsApp transfer data to Facebook?". This post asks a simple question: has WhatsApp really stopped sharing your personal data with Facebook following these complaints in the European Union? Unfortunately no, even if the answer is not easy to find.
Here is what the WhatsApp privacy policy says for residents of the European Union (updated April 2018):
How we work with other Facebook entities
We are part of the Facebook entities. As a member of the Facebook entities, WhatsApp receives information from Facebook entities and also transmits information to them. We may use the information they send to us, and they may use the information we send to them, to help us operate, provide, improve, understand, personalize, support and market our Services and their offerings [...].
By delving deeper into the question through the article "How we work with Facebook entities", we understand that personal data exchanges between WhatsApp and Facebook are very numerous. One point, however: without the European Commission's action, Facebook would go even further in processing your personal data:
We do not share data in order to improve Facebook products on the platform and to provide better advertising experiences on Facebook.
Today, Facebook does not use your WhatsApp account information to improve your experience with Facebook products or to provide you with more targeted Facebook ads on its platform. This is the result of discussions with the Irish Data Protection Commissioner and other data protection authorities in Europe. We're always looking for new ways to improve your experience with WhatsApp and the Facebook Business Products you use. If in the future we decide to share such data with the Facebook Entities for this purpose, we will first enter into an agreement with the Irish Data Protection Commissioner to establish a mechanism that allows such use. We will inform you of new experiences made available to you and our practices regarding the use of your data.
And indeed, if you are not a resident of the European Union, Facebook holds nothing back:
Facebook and other entities in the Facebook family may also use information we provide to improve your experiences within their services, such as making product suggestions (e.g. from friends, connections or interesting content) and displaying relevant offers and advertisements.
Since early January, the latest update to WhatsApp's terms of use has gone down badly. The date when the new terms take effect has been postponed: initially scheduled for February 8, it will now be May 15, giving WhatsApp time to fine-tune its communication.
While sharing your personal data with Facebook had already been in effect for almost 5 years, this update will allow Facebook to go further:
- Sharing your WhatsApp transactional data with the "Facebook Shops".
- Sharing your communications with businesses, via a communications hosting service offered by Facebook.
This hosting service is a first breach in end-to-end encryption, because these communications will not be encrypted at the host (one of the two “ends” of the encryption). Here is how WhatsApp muddies the waters:
WhatsApp considers chats with businesses using the WhatsApp Business app or managing and storing customer messages themselves to be end-to-end encrypted. Once the message is received, it will be subject to the company's own privacy practices. The company can allow a certain number of employees, or even other service providers, to process the message and respond to it.
Some businesses may choose to store and respond to customer messages securely through WhatsApp's parent company, Facebook. You can always contact companies to learn more about their privacy practices.
For convenience, the vast majority of companies will probably choose Facebook's hosting solution. The content of your conversations with companies will therefore be visible to the “host”, Facebook, and subject to its privacy practices.
So the last argument in WhatsApp's favor (especially compared with Telegram) is that your personal messages remain protected by end-to-end encryption by default:
The privacy and security of your personal messages and calls remains unchanged. They are protected by end-to-end encryption and neither WhatsApp nor Facebook can read or listen to them. We will never weaken this security and we feature this information in every chat so you can be aware of our commitment.
Note that WhatsApp uses Signal's open source protocol for message encryption, which is a strong point:
Every WhatsApp message is protected by the same Signal encryption protocol, securing messages before they leave your device.
But for the rest of your personal data, anything goes. How could WhatsApp betray its users to this extent? For a detailed history of WhatsApp's compromises with (and resistance to) its parent company, read the excellent Forbes article "Exclusive: WhatsApp Cofounder Brian Acton Gives The Inside Story On #DeleteFacebook And Why He Left $850 Million Behind". The WhatsApp co-founder says in particular:
I sold my users’ privacy to a larger benefit. I made a choice and a compromise. And I live with that every day.
Brian Acton left WhatsApp in September 2017 (and his fellow co-founder Jan Koum in April 2018). Today he is the head of the Signal Foundation, launched in February 2018 with $50 million in seed funding from Acton. He also has an important operational role at Signal.
Facebook has since driven away Instagram co-founders Kevin Systrom and Mike Krieger, over a story of jealousy (Instagram is fashionable, Facebook less so) and growth at all costs (degrading the Instagram app to highlight the Facebook app).
A happy ending?
As we can see, WhatsApp follows the model of its parent company: it established itself by being more respectful of privacy than its competitors. Bought by Facebook and now in a dominant position, it betrays your trust by gradually eroding your privacy.
Will it get away with it? That will depend on our collective actions. For my part, the task is even harder than with Facebook and Instagram (accounts I was able to close some time ago). I have been able to migrate some conversations and groups to Signal, but I will have to convince a lot of people before I can hope to close my WhatsApp account.
WhatsApp Privacy on the App Store.
Signal Privacy on the App Store.
Still, there is reason to be optimistic in the long term. Unlike Facebook or Instagram, which today have no real competitors, messaging services like Telegram and Signal will give WhatsApp a hard time. And the current wave of Signal installs is a pleasure to see.
A tweet by Elon Musk can never hurt.
We've had worse sponsors.
