Synthetic media technologies—popularly known as deepfakes—have real potential for positive impact. Voice synthesis, for example, will allow us to speak in hundreds of languages in our own voice. Video synthesis may help us simulate self-driving-car accidents to avoid mistakes in the future. And text synthesis can accelerate our ability to write both programs and prose.
But these advances can come at a gargantuan cost if we aren’t careful: the same underlying technologies can also enable deception with global ramifications.
Thankfully, we can both enable the technology’s promise and mitigate its peril. It will just take some hard work.
What I’m arguing here is meant to be a call to action to do that work—and a guide to support those of us creating such technology, whether we’re doing ground-breaking research, developing new products, or just making open-source systems for fun. This is also for those investing in or funding such work, the journalists who might help ensure that technology creators take their impacts seriously, and the friends or family of those creating this tech.
We can longer absent ourselves of responsibility by saying “There’s nothing we can do if people misuse our tools.” Because there are things we can do; it’s just that we often don’t bother. Another argument, “This technology will get made anyway,” is not entirely false—but the how and when matter significantly, and can change as a result of our choices. (For more on this, including some threat modeling, please check out our full paper, especially Section 2 about what leads to harm.) Finally, we can’t hide behind the trite acknowledgement that “there has always been deception” while ignoring the significant differences in degree and impact.
The costs of deepfake technology are not just theoretical. Synthetic face-swap videos harass journalists into silence; synthetic voices are being used for large fraudulent transactions, and synthetic faces have allegedly supported espionage. All of that is in spite of the current challenges of using hacked-together beta-quality software. The obstacles to using synthetic media are still too high for the technology to be compelling to most malicious actors—but as it moves from buggy betas and into the hands of billions of people, we have a responsibility to avoid worst-case scenarios by making it as hard as possible to use deepfakes for evil. How?
There are several things we can do that make malicious use much less likely. One obvious, simple, and effective approach is to carefully vet those who can use a tool. This is what companies like Synthesia are doing—essentially, working only with vetted enterprise clients.
An alternative is to constrain usage: to limit what users can synthesize or manipulate. For example, it is possible to build tools to ensure that only particular pre-chosen voices or faces can be manipulated. This is what Humen, for example, is doing—providing a limited set of movements that the person in a generated video can make.
These may not be an option for many systems, however. So what else can one do?
For synthetic media tools that are general and may be made widely available, there are still many possible ways to reduce malicious use. Here are some examples.
Not all these strategies are applicable to every system. Some may have their risks, and none are perfect—or sufficient on their own. They are all part of a “defense in depth,” where more is more. Even if we have vetting or constraints, these approaches still make a system more robust against adversaries. And while following these rules might work best for “software as a service” systems, which are delivered without revealing any source code, they could still provide some value for open-source tools and models: many bad actors won’t have the technical capability to get around these protective measures. (The full paper explores when to publish source code in the first place).
Doing extra work to protect people from harm can be hard to justify in today’s competitive business environment—until an irreversible catastrophe happens. So how do we help ensure that these approaches are implemented before it’s too late? Here are four things funders, governments, journalists, and the public can do now to support those making ethical synthetic media tools.
That means that we must invest in research into all these areas so that we have widely available, well-funded, open-source tools to implement these approaches. The history of information security shows that when easy-to-use open-source tools can be used to keep things secure, a lot more things end up secure. The same logic applies here: at the very least, we urgently need to make it easy to provide standardized disclosure metadata, watermarks, and logging. We also need research to explore whether it is feasible to bake detectability into trained models before distribution. Without this kind of infrastructure and research, we will see many well-intentioned new tools being used in horrific ways.
Just as in privacy and security, we must support communities focused on hardening systems and addressing bad actors, and companies must pay people to do this work, either as consultants or in house.
If a company or tool doesn’t at least attempt to reduce the chance of misuse, funders and developers should not support it. Tool makers that do not implement the best practices listed above must have very good reasons, and app stores should require them by default.
If they are negligent, we should be calling them out—even if they are our investees, coworkers, friends, or family. We can create ethical norms by commending those who do the right thing, and pushing those who don’t to do better.
Organizations advancing deepfake technology, like Nvidia, Adobe, Facebook, and Google, should be investing heavily in all of the above. Venture capitalists and foundations should also do their part in supporting this work and being careful who they support.
This is only one slice of a much broader set of efforts that are needed, and in many cases they may buy us only a little more time. That means it’s imperative to ensure that platforms and policymakers use that time wisely to make our information ecosystem more resilient.
Our past is littered with those who wished they had introduced their inventions more carefully—and we only have one future. Let’s not screw it up.
—Aviv Ovadya is the founder of the Thoughtful Technology Project and a non-resident fellow at the German Marshall Fund’s Alliance for Securing Democracy. See the full paper on mitigating negative impacts of synthetic media research, coauthored with Jess Whittlestone, here.