Introducing Sama Red Team: Boosting Safety and Reliability for Generative AI Models

GenAI is in its infancy. It’s easier to break, it's less robust than other models, and it’s easier to go around safe guards. Red-teaming in early stages can help technologies be as safe as possible while driving the responsible development and usage of AI.

Sama Red Team, which is made up of ML engineers and applied data scientists, evaluates a model's fairness, safeguards‌ and compliance with laws by digging into text, image, voice search‌ and more to identify and fix any issues before they turn into vulnerabilities.

Red-teaming has a larger impact beyond improving model security: it helps build responsible and resilient AI models. Duncan Curtis, SVP of AI product and technology at Sama, said, “Although ensuring that a model is as secure as possible is important to performance, our teams’ testing is also crucial for the development of more responsible AI models.” 

Read more: Duncan Curtis goes into greater detail about red-teaming for GenAI with VentureBeat's Alex Perry.

In a world of deep fakes and data breaches, a model’s creator is ultimately responsible for its outputs. Red teaming professionals have deep knowledge of the complexities of generative AI models' inner workings, and can proactively uncover vulnerabilities that may generate offensive content or reveal personal information. Acting as adversaries, Sama Red Team will simulate real-world scenarios, mimic cyberattacks, and test for compliance. When they “trick” your model, you can then correct it before it launches to the public or your customers.

We believe the only way this process brings value is if it considers your context and assumptions about your models in order to set the right targets around threats that matter most to you. Sama’s red teaming projects start with tailored consultations to understand requirements for model performance, then focus on four key areas—fairness, privacy, public safety, and compliance. Based on the results, the team will fine-tune prompts or create new ones to further probe the vulnerability, with the ability to also create large-scale tests for additional data.

And if you need to scale up, our team of 4,000+ highly-trained annotators is ready to step in and grow with your projects. These teams of full-time annotators have an average tenure of 2-3 years, and receive specific training in model validation. We’re proud to invest in upskilling our teams, and invest in training them to handle even the most complex model data. Learn more here. 

Sama Red Team continues to stay on top of the latest trends and testing techniques to identify the most effective ways to trick generative AI models and expose vulnerabilities. Learn more here.