Tremau Turns 5: How We See Trust & Safety Today

Since Tremau was founded in 2021, the environment T&S teams operate in has become significantly more complex. Regulation has accelerated. AI has changed both how harmful content is created and how platforms respond to it. The role of T&S inside organizations is evolving. And the work itself increasingly requires a combination of policy and compliance expertise, technical understanding, data and cross-functional collaboration.

So, for our fifth anniversary, we asked some of Tremau’s leaders and subject-matter experts across T&S, compliance, product, technology and operations to share one thing they believe is particularly important to understand about this field today. Their perspectives are different, but together they show how much the scope of T&S has expanded – becoming more technical, more regulated, more closely connected to business priorities, and more complex to operate.

When we founded Tremau five years ago, one of our core convictions was that transparency was essential to build trust in technology.

But as we started working with platforms, we quickly saw how difficult meaningful transparency actually was. The problem was not simply a lack of reporting. It was that Trust & Safety systems often involved multiple components: tools for detection, user reports, moderation and appeals, with policy, enforcement and reporting managed across different systems and teams.

That complexity reflected the evolving nature of T&S. The focus had essentially been on one part of the problem: detecting harmful content (better classifiers). Detection matters, of course, but the key question became: once something is detected, what happens next? Can you show which policy applied, why a decision was made, whether similar cases were treated consistently, how an appeal changed the outcome, or how a policy update affected enforcement across the platform?

Answering those questions requires much more than a detection layer. It requires the different parts of T&S to work as a coherent and orchestrated system, connecting policy, AI, human decisions, enforcement, appeals and data – at scale.

Regulation has made that need much more visible. Transparency, appeals, reporting to authorities … is no longer simply a matter of good practice; in many cases, platforms are expected to demonstrate how their systems work and provide evidence that their processes are consistent and accountable.

That is one of the biggest changes in the past five years: The real challenge is building the infrastructure that allows platforms to manage risk consistently, transparently and at scale. The platforms that earn lasting trust will be those that treat this infrastructure not as a back-office function, but as the foundation of every decision they make.

The next chapter of Trust & Safety will be defined not only by better models, but by better systems: infrastructure that connects policy to action, enables automation at scale, makes decisions explainable and turns accountability into something platforms can demonstrate every day.

As T&S systems become more sophisticated, so does the work of the people running them. AI creates unprecedented opportunities to operate at greater scale, but implementing it well still requires substantial human expertise.

AI gives T&S opportunities to scale like never before. But much of what we see still requires significant human expertise and investment. It’s not “plug and play” yet.

From using LLMs to support policy development, to calibrating and testing how those policies work in practice, to launching AI agents, each step requires thoughtful expert input. We have seen quality gaps, over-enforcement and failures to understand market nuances.

The field is moving fast, and teams doing this well are layering AI into a broader safety framework rather than treating it as a standalone solution. And perhaps this is what strikes me the most: our industry is small and still not well understood. Many continue to conflate T&S with “moderation.”

That old trope should be retired. The job today is data-driven, requires an understanding of technical solutions, demands critical thinking and the balancing of competing interests, and is anchored by the need to do the right thing for both the platform and the user. This is so much more than a basic content review flow and requires multi-skilled teams and tools.

Policy is one area where that complexity becomes particularly visible. Human moderators have historically compensated for ambiguity and inconsistency in ways that automated systems cannot and AI can scale policy weaknesses just as efficiently as it scales good decisions.

Only the biggest platforms have people whose actual job is writing policy. Everywhere else it is done by the T&S team, alongside escalations, reporting and everything else. Some become very good at it.

Even so, the text gets adjusted by a moderation lead here and a QA manager there, and what you end up with is a patchwork. It works because human moderators fill the gaps with their own judgment, and nobody sees that happen.

AI agents do not do that repair work. You can teach an agent to escalate when it is unsure. It has no reason to be unsure when a guideline is clear and wrong.

There is also a second document now. A policy is written for a human reader and leaves room for interpretation. A system prompt has to turn that same policy into explicit instructions a model can act on, and it changes from one model to another. Writing one is a skill of its own, and the people who write prompts are not always the people who know the policy.

An agent goes through the same path as a moderator. A moderator is trained, then reviewed in production. An agent is tuned on cases you already know are correct, then monitored with the same quality checks your team already runs.

What changes is the volume. A moderator applies a bad rule a hundred times a day. An agent applies it to everything.

AI is changing not only how platforms run T&S, but also the threats they are trying to manage. The same generative technologies available to safety teams are also available to those trying to evade them.

Tremau has witnessed the shift before and after the emergence of large language models, which introduced an unprecedented set of Trust & Safety challenges.

Threat velocity has accelerated rapidly as malicious actors leverage generative tools to increase their output, producing vast amounts of harmful or illegal content and developing increasingly sophisticated ways to circumvent traditional static guardrails.

T&S defenses therefore need to become more dynamic too, adapting as abuse patterns and circumvention techniques change. Static guardrails and rudimentary detection are increasingly poorly suited to threats that can evolve this quickly.

There is no single technical answer. The challenge is building T&S infrastructure that can evolve alongside the threats it is designed to address.

At the same time, T&S teams are being asked to operate these increasingly complex systems under much greater external scrutiny. The regulatory environment surrounding online safety has expanded dramatically over the same five-year period.

When Tremau started five years ago, the DSA and the Online Safety Act had not yet taken effect. There were around thirty online safety laws globally, built up over two decades. In the last five years, we have added forty more: double the previous decade, in half the time, with no signs of slowing down. Just last month, the Commission proposed another landmark law EU KIDS Act, with new obligations on age assurance and safety by design.

And these laws have teeth. There are over fifty ongoing investigations across the European Commission, Digital Services Coordinators and Ofcom, and the Commission has already issued several high-profile fines against large platforms with Australia, Brazil and Indonesia not far behind.

So if your T&S team is feeling overwhelmed, it’s not you. The job has fundamentally changed. T&S is now a firmly regulated industry, and each new law adds a layer of scrutiny to the existing safety and product development infrastructure. Regulators across the globe want better data on your service, continuous monitoring of risks on your platform and proof that safety is a core metric of design decisions.

The complexity of the digital world means the way we manage governance, risk and compliance has to evolve. We often look to financial services as the benchmark, but I believe the online world needs something even more sophisticated. Regulation is growing, new harms emerge every day, AI is accelerating the pace of change, and bad actors are becoming more capable. And ultimately, we are dealing with humans interacting with humans. There is very little that is more complex than that.

The answer cannot be more standalone compliance processes or a once-a-year exercise. Compliance has to become part of business as usual, connecting regulation and policy with risk, product, technology and T&S operations. That means bringing together signals from inside and outside the organisation, understanding what they mean for risk, translating that understanding into everyday decisions and actions, and continuously measuring whether the controls we put in place are actually working. Technology can connect those pieces and make that feedback loop possible at scale.

All of this also changes the role T&S plays inside an organization. As its responsibilities expand across technology, risk, regulation and user experience, its impact increasingly extends beyond the safety function itself.

The perception of Trust & Safety within organizations is undergoing a fundamental shift: from a traditional cost center toward a core function that creates value across multiple dimensions.

T&S operations increasingly create value well beyond the safety function itself. For example, T&S teams can train Sales on baseline safety narratives so they can answer advertiser questions directly, helping protect revenue and partnership targets. They can also support product expansion in key markets such as Brazil, the EU, Indonesia, etc.

Under-investing in T&S therefore doesn’t just limit safety capabilities; it can have knock-on effects across the wider organization. Making this value visible and measurable is an important part of elevating T&S internally.

Taken together, these perspectives show a function that is expanding in almost every direction.

T&S teams are expected to operate increasingly complex systems, integrate rapidly evolving technologies, translate policy into consistent decisions, respond to changing threats, meet growing regulatory expectations and demonstrate their value to the wider business.

None of these challenges exists in isolation. Technology affects policy. Policy shapes enforcement. Enforcement generates the data needed for quality assurance, risk management and compliance. And the quality of that entire system ultimately affects users, regulators and the business itself.

That interconnectedness is perhaps the clearest lesson from Tremau’s first five years: effective Trust & Safety increasingly depends on how well the different pieces work together.

As Tremau enters its next chapter, that is the problem we remain focused on solving.