AI/ML security assessment

In today’s rapidly evolving digital market, delivering the best products and services to your clients is crucial. That’s why AI and ML are omnipresent. They are transforming industries and offering new opportunities for businesses. 

However, as these technologies advance at a rapid pace, new security threats and incidents emerge almost daily, including issues related to ML, AI, LLMs, and deepfakes. It’s a new challenge for many businesses – to stay ahead of vulnerabilities and keep their clients’ data secure. 

Many questions are to be asked: 

• How can you be an early adopter of AI or ML while ensuring security?  
• Are you ready to address the security aspects of your AI/ML integration?  
• Do you recognize the potential risks? 

Why choose AI/ML security assessment 

If you’re considering incorporating AI/ML into your service or product, this assessment is for you. Our AI/ML security assessment covers everything from analyzing your business case and identifying risks to executing test cases on your implemented solution. 

Our approach to AI/ML security assessment

AI/ML business case analysis  

The journey starts with gaining a deep understanding of the business case that led you to the adoption of AI/ML. This enables us to identify any business-related threats specific to you, ensuring that your investment in AI/ML is secure and beneficial. 

At this stage, we will be asking questions like: 

• What business problems are you solving with AI/ML? 
• What business threats have you identified? 
• What are the worst possible risks of using AI/ML for your business? 

AI/ML technical overview 

Next, we dive into the technical details of your solution, analyzing everything from its architecture to the specific components and services you use. This detailed analysis ensures that we understand your system inside and out, allowing us to pinpoint any vulnerabilities and strengthen your overall security. 

At this stage, we will be asking questions about: 

• The data on which the solution is based 
• The architecture of the components in both the backend and frontend layers 
• The relationships between the components within the solution 
• How you integrated the AI/ML components into your existing architecture 
• Your approach to authentication, authorization, and data validation when interacting with the AI/ML components 

AI/ML Threat Modeling 

With this comprehensive understanding, we conduct AI/ML threat modeling to identify any risks that could impact your application or platform. By identifying these risks early, we help you avoid potential disruptions and maintain the trust of your customers. If you wish to learn more about how we do Threat Modeling, read this article. 

AI/ML security assessment 

Once we’ve identified the risks, we perform an AI/ML security assessment to determine if these threats could affect your solution. This proactive approach ensures that any issues are addressed before they can impact your business, providing you with peace of mind. 

Since your use case is likely embedded in a web or mobile application, we recommend testing the entire solution, not just the AI/ML components. From our experience, final reports often reveal vulnerabilities in the AI/ML components as well as in API, web application, or mobile application. 

Report 

By starting with a deep dive into your business and technical setup and conducting thorough Threat Modeling, we ensure our AI/ML security assessment and final report address all potential issues. This means you receive accurate prioritization of risks and tailored recommendations that help you protect your company and enhance your security. 

How we work – examples 

LLM – Large Language Model 

If we identify the use of a Large Language Model (LLM) in your solution, such as an interactive chat or assistant, we will evaluate several key aspects: 

• Can it produce undesirable outcomes, such as swearing or mocking? (validation bypass) 
• Can it be manipulated to respond in unintended ways? 
• Can the prompt be altered to produce unintended results, such as through prompt injection or jailbreak techniques like DAN (e.g., the Air Canada incident)? 
• Can details about the underlying model be exposed? (model theft) 
• Do AI/ML-based mechanisms require significant computing resources that could impact the performance of the entire web/mobile application, increasing page load times and degrading user experience? 

Facial recognition / Biometrics 

If your solution addresses the question of distinguishing between a real person in a picture/video stream and a fake one, our evaluation questions will include:  

• Did you select a training dataset that includes appropriate data, without excluding based on gender or race?  
• How does your solution manage various image and video formats?  
• What methods does your solution use to conduct a liveness check?  
• How does your solution handle images with multiple faces?  
• How does your solution address challenges such as glasses, beards, different hairstyles, etc.?  
• How does your solution cope with low-quality video or images?  
• How does your solution process images with multiple layers?  
• How does your solution manage animated GIFs or PNGs?  
• How does your solution store ID cards (if needed for verification)?  
• Does your solution resort to human verification if the software behaves incorrectly?  

To assess the resilience of your biometrics model to potential attacks, we test it through various scenarios. They range from basic tricks to more complex test cases reflecting real-world threats. Moreover, we make AI/ML techniques simulate potential attackers that are attempting to bypass security measures using AI-generated video or audio samples of a targeted victim. 

Here, you can find our articles dealing with biometrics security: 

1. Before you implement Face Recognition to your app – AI hack study
2. Voice Biometrics – how easy is it to hack them with AI Deepfake? 
3. Attacking the face recognition authentication – how easy is it to fool it?

With a bit of your help, we’ll conduct a comprehensive AI/ML security assessment. This will pinpoint vulnerabilities and their potential impact on your business. The result? A fortified AI/ML system and substantial cost savings. Proactive measures today mean fewer expensive fixes tomorrow. 

Why us? 

For over 20 years, Securing has been a trusted partner in the area of IT security. Our highly specialized team brings expertise across various domains, including web, mobile, cloud, embedded systems, IoT, AI/ML, and blockchain. Our goal is to detect vulnerabilities that could be exploited by intruders. We focus on finding defects early to enhance your security.  

As AI/ML security is an emerging and rapidly evolving field, we dedicate our time to staying current with new developments like OWASP TOP 10 Large Language Model Applications and MITRE Atlas. This commitment to staying up-to-date means you benefit from the latest security insights and best practices, ensuring your solution is protected against the newest threats. 

During AI/ML security assessments we provide: 

• Deep dive into your business case and technical details, 
• Identification of threats relevant to your case, 
• Detailed report with recommendations, 
• Team with experience in AI/ML use and security aspects. 

Besides a few AI/ML security assessments already behind us, our specialists have found more than 10 AI/ML-related CVEs during bug bounties in their spare time.  

DORA compliance 

AI/ML security assessment can contribute to the compliance with DORA principles. It aligns with DORA’s emphasis on risk management. 

How to get a quote for your project? 

Fill out our contact form, indicating that you are interested in AI/ML security assessment. We will get back to you soon with our full offer.

You can also schedule a short discovery call straight away by selecting a time slot in the calendar below: