Skip to main content

Get the Reddit app

Scan this QR code to download the app now
Or check it out in the app stores

r/redditsecurity

members
online


Q4 2023 Safety & Security Report ADMIN Q4 2023 Safety & Security Report

Hi redditors,

While 2024 is already flying by, we’re taking our quarterly lookback at some Reddit data and trends from the last quarter. As promised, we’re providing some insights into how our Safety teams have worked to keep the platform safe and empower moderators throughout the Israel-Hamas conflict. We also have an overview of some safety tooling we’ve been working on. But first: the numbers.

Q4 By The Numbers

Category Volume (July - September 2023) Volume (October - December 2023)
Reports for content manipulation 827,792 543,997
Admin content removals for content manipulation 31,478,415 23,283,164
Admin imposed account sanctions for content manipulation 2,331,624 2,534,109
Admin imposed subreddit sanctions for content manipulation 221,419 232,114
Reports for abuse 2,566,322 2,813,686
Admin content removals for abuse 518,737 452,952
Admin imposed account sanctions for abuse 277,246 311,560
Admin imposed subreddit sanctions for abuse 1,130 3,017
Reports for ban evasion 15,286 13,402
Admin imposed account sanctions for ban evasion 352,125 301,139
Protective account security actions 2,107,690 864,974

Israel-Hamas Conflict

During times of division and conflict, our Safety teams are on high-alert for potentially violating content on our platform.

Most recently, we have been focused on ensuring the safety of our platform throughout the Israel-Hamas conflict. As we shared in our October blog post, we responded quickly by engaging specialized internal teams with linguistic and subject-matter expertise to address violating content, and leveraging our automated content moderation tools, including image and video hashing. We also monitor other platforms for emerging foreign terrorist organizations content to identify and hash it before it could show up to our users. Below is a summary of what we observed in Q4 related to the conflict:

  • As expected, we had increased the required removal of content related to legally-identified foreign terrorist organizations (FTO) because of the proliferation of Hamas-related content online

    • Reddit removed and blocked the additional posting of over 400 pieces of Hamas content between October 7 and October 19 — these two weeks accounted for half of the FTO content removed for Q4

  • Hateful content, including antisemitism and islamophobia, is against Rule 1 of our Content Policy, as is harassment, and we continue to aggressively take action against it. This includes October 7th denialism

    • At the start of the conflict, user reports for abuse (including hate) rose 9.6%. They subsided by the following week. We had a corresponding rise in admin-level account sanctions (i.e., user bans and other enforcement actions from Reddit employees).

    • Reddit Enforcement had a 12.4% overall increase in account sanctions for abuse throughout Q4, which reflects the rapid response of our teams in recognizing and effectively actioning content related to the conflict

  • Moderators also leveraged Reddit safety tools in Q4 to help keep their communities safe as conversation about the conflict picked up

    • Utilization of the Crowd Control filter increased by 7%, meaning mods were able to leverage community filters to minimize community interference

    • In the week of October 8th, there was a 9.4% increase in messages filtered by the modmail harassment filter, indicating the tool was working to keep mods safe

As the conflict continues, our work here is ongoing. We’ll continue to identify and action any violating content, including FTO and hateful content, and work to ensure our moderators and communities are supported during this time.

Other Safety Tools

As Reddit grows, we’re continuing to build tools that help users and communities stay safe. In the next few months, we’ll be officially launching the Harassment Filter for all communities to automatically flag content that might be abuse or harassment — this filter has been in beta for a while, so a huge thank you to the mods that have participated, provided valuable feedback and gotten us to this point. We’re also working on a new profile reporting flow so it’s easier for users to let us know when a user is in violation of our content policies.

That’s all for this report (and it’s quite a lot), so I’ll be answering questions on this post for a bit.


Q3 2023 Safety & Security Report ADMIN Q3 2023 Safety & Security Report

Hi redditors,

As we come to the end of 2023, we’re publishing our last quarterly report in this year. In this edition, in addition to our quarterly numbers, you’ll find an update on our advanced spam capabilities, product highlights, and a welcome to Reddit’s new CISO.

One note: Because this report reflects July through September 2023, we will be sharing insights into the Israel-Hamas conflict in our following report that covers Q4 2023.

Now onto the numbers…

Q3 By The Numbers

Category Volume (April - June 2023) Volume (July - September 2023)
Reports for content manipulation 892,936 827,792
Admin content removals for content manipulation 35,317,262 31,478,415
Admin imposed account sanctions for content manipulation 2,513,098 2,331,624
Admin imposed subreddit sanctions for content manipulation 141,368 221,419
Reports for abuse 2,537,108 2,566,322
Admin content removals for abuse 409,928 518,737
Admin imposed account sanctions for abuse 270,116 277,246
Admin imposed subreddit sanctions for abuse 9,470 1,130
Reports for ban evasion 17,127 15,286
Admin imposed account sanctions for ban evasion 266,044 352,125
Protective account security actions 1,034,690 2,107,690

Mod World

In December, Reddit’s Community team hosted Mod World: an interactive, virtual experience that brought together mods from all around the world to learn, share, and hear from one another and Reddit Admins. Our very own Director of Threat Intel chatted with a Reddit moderator during a session focused on spam and provided a behind-the-scenes look at detecting and mitigating spam. We also had a demo of our Contributor Quality Score & Ban Evasion tools that launched earlier this year.

If you missed Mod World, you can rewatch the sessions on our new Reddit for Community page, a one-stop-shop for moderators that was unveiled at the event.

Spam Detection Improvements

Speaking of spam, our team launched a new detection method to assess content and user-level patterns that help us more decisively predict whether an account is exhibiting human or bot-like behavior. After a rigorous testing period, we integrated this methodology into our spam actioning systems and are excited about the positive results:

  • We identified at least an additional 2 million spam accounts for enforcement

  • Actioned 3x more spam accounts within 60 seconds of posting a post or comment

These are big improvements to how we’re able to keep spam off the site so users and mods never need to see or action it.

What’s Launched

Reports & Removals Insights for Communities

Last week, we revamped the Community Health page for all communities and renamed it “Reports & Removals.” This updated page provides mods with clear and new insights around content moderation in their communities, including data about Admin removals. A quick summary of what changed:

  • We renamed the page to “Reports and Removals” to better describe exactly what you can find on the page.

  • We introduced a new “Content Removed by Admins” chart which displays admin content removals in your community and also distinguishes between spam and policy removals.

  • We created a new Safety Filters Monthly Overview to help visualize the impact of Crowd Control and the Ban Evasion Filter in your community.

  • We modernized the page’s interface so that it’s easier to find, read, and tinker with the dashboard settings.

You can find the full post here.

Simplifying Enforcement Appeals

In Q3, we launched a simpler appeals flow for users who have been actioned by Reddit admins. A key goal of this change was to make it easier for users to understand why they had been actioned by Reddit by tying the appeal process to the enforcement violation rather than the user’s sanction.

The new flow has been successful, with the number of appealers reporting “I don’t know why I was banned” dropping 50% since launch.

Reddit’s New CISO

We’re happy to share that a few months back, we welcomed a new Chief Information Security Officer: Fredrick Lee, aka Flee (aka u/cometarystones), officially the coolest CISO name around! He oversees our Security and Privacy teams and you may see him stop by in this community every once in a while to answer your burning security questions. Fun fact: In addition to being a powerlifter, Flee also lurks in r/MMA, so bad folks better watch out.