What does that tattoo mean? AI app decodes extremist political symbols

You probably won’t have a problem recognizing the Confederate flag or the LGBTQ rainbow flag. But the Internet’s many subcultures and communication channels offer an endless amount of new symbols, and you may not know what to think about a Molon Labe patch or an igloo flag when you are at a protest or see a news story about one.

For this reason, Columbia University researchers have developed an app called VizPol that can be used to recognize such symbols artificial intelligence and the camera of a phone.

VizPol works along the lines of Cornell University’s Merlin Bird ID, which uses AI to identify birds on your phone’s photos. The underlying AI technology, sometimes referred to as machine learning or deep learning, cannot do everything a human brain can. But it’s really good to spot patterns, whether they’re on a Kekistani flag at a far-right rally or on a Bohemian waxwing in a tree.

VizPol is not for the general public or for content moderators on Facebook or Twitter, at least not yet. It is aimed at journalists who may need to understand obscure or cryptic symbols on flags, t-shirts, signs, tattoos, or badges, said Susan McGregor, assistant professor of journalism and deputy director of the Tow Center for Digital Journalism, a research and development agency Teaching center at the Columbia Journalism School.

Understanding these symbols can be important for professionals whose job it is to provide context during an increase in protest activity, such as armed anti-lockdown protesters entering a statehouse or activists who, after George Floyd’s death, opposed the brutality of the Police march. A well-founded assessment can be crucial to understand raw videos uploaded to Twitter or to sort out disinformation campaigns on Facebook.

Journalists need to understand what the symbols in a photo or video mean before using the images. “The problem is that finding out is extremely labor intensive,” said McGregor.

What did this tattoo mean?

The project started about a year and a half ago when Nina Berman, professor of photojournalism at the Columbia Graduate School of Journalism, discovered symbols that could be important at rallies and protests. At the 2018 Unite the Right 2 rally, she photographed a quietly spoken woman who said she was protesting the threat to her First Amendment rights, McGregor said. It was only later that Berman discovered “1488” tattooed on the woman’s arm and realized that it was a code for a 14-word slogan by the white Supremacists and a reference to the letter H in “Heil Hitler” . (“H” is the eighth letter in the alphabet.)

Now VizPol can recognize many other symbols, many of which are used by different white nationalist sub-groups. The symbols include the Odal rune, a figure used by a particularly brutal NS-SS regiment. the black igloo, a drawing of the right-wing extremist Boogaloo movement; and a colonial-era U.S. flag modified to represent the Three Percenters, an arms rights group. VizPol can also recognize the Nazi-stylized flag of Kekistan, a fictional country created by members of the online 4chan forum, as a criticism of the perceived excessive political correctness.

Brian Levin, director of the Center for the Study of Hate and Extremism at California State University in San Bernardino, said the app will help journalists and researchers even with difficult problems like misidentifying or not identifying a symbol. When automation works, journalists and researchers can keep up with symbols that are changing at internet speed.

“Memes and symbols are constantly being co-opted, vomited and abandoned by extremists,” said Levin.

Icon decryption

Such symbols are intentionally dark. “The intention is to signal a known section of the community,” said McGregor. VizPol can also help clarify which subgroups of a movement a person might belong to, such as the Patriot Front, the American Identity Movement or the National Socialist Movement.

VizPol doesn’t just focus on the images of right-wing groups. It also recognizes symbols used by Antifa, a loose collection of protest groups that deal with neo-Nazis and white Supremacists. The symbols used by Antifa, short for antifascists, are often straightforward, e.g. B. three arrows or an image overlapping red and black flags.

The murder of George Floyd, an unarmed black man by a white policeman, on May 25 and the subsequent protests against Black Lives Matter once again brought Antifa to the news. This has made the movement’s symbols relevant to the VizPol database and prompted developers to look for more additions, McGregor said.

VizPol’s AI brains

The researchers have made the app available to more than 100 people for testing, says Ishaan Jhaveri, a researcher who wrote the app together with PhD student Bhaskar Ghosh. Postdocs Svebor Karaman, Xu Zhang and Guangxing Han set up VizPol’s machine learning system.

A particular challenge for VizPol is to find enough images to train the AI ​​symbol recognition system, especially if these symbols are distorted in the real world by t-shirt folds or partially hidden by another symbol.

To work around this problem, the researchers generated synthetic versions of the images using the Blender 3D imaging software. For example, the team mapped symbols onto simulated wavy flags. The team is also working on an update to automatically recognize symbols. Training data is used in which the researchers added them to photos in a UCLA collection of protest photos.

Users can add new symbols

Users of the app also contribute to its benefits. After a user takes a photo, the app analyzes it and suggests possible identifications. There is also a web interface that is useful for image editors sitting on their PCs. Users can provide feedback on whether the app has been judged correctly. They can also upload their own images to update VizPol’s symbol recognition capabilities. Columbia researchers, however, review each submission and add descriptions of the meaning of the symbols.

The app is currently being tested with a relatively small group to protect itself from misuse problems such as uploading fake photos or over trusting VizPol’s judgment in a world where symbols can have many meanings, some of which are harmless.

“We hope to be able to work with news projects and organizations,” said McGregor, including photo agencies that distribute many pictures and captions. “We want to help journalists do what they do. It’s an advantage for people everywhere.”

Black lives count. visit blacklivesmatter.carrd.co to learn how to donate, sign petitions, and protest safely.

Currently running:
Look at that:

How to protect your phone (and your privacy) during a protest

4:31