Vertex AI Flaws Allow ML Model Exfiltration and Privilege Escalation

Cybersecurity researchers have identified two high-severity vulnerabilities in Google's Vertex AI platform that could enable attackers to escalate privileges and exfiltrate machine learning (ML) models from the cloud. These flaws, discovered by Palo Alto Networks Unit 42, highlight significant risks to sensitive data in AI environments.

Flaw Overview

Vertex AI, launched in May 2021, is Google's platform for deploying and managing custom ML models. The vulnerabilities stem from the improper handling of custom job permissions and model deployment processes, exposing users to potential privilege escalation and data exfiltration.

1. Privilege Escalation via Vertex AI Pipelines

The first flaw exploits a feature called Vertex AI Pipelines, used to automate ML workflows. By crafting a malicious custom job, attackers can deploy an image that launches a reverse shell, granting unauthorized access to the environment.

This attack leverages the extensive permissions of the service agent account, including access to:

1. Service accounts
2. Storage buckets
3. BigQuery tables

Attackers can abuse these permissions to infiltrate internal Google Cloud Platform (GCP) repositories and download container images.

2.Model Exfiltration via Poisoned Models

The second vulnerability involves deploying a poisoned model to the Vertex AI environment. When executed, the model creates a reverse shell, allowing attackers to:

1. Enumerate Kubernetes clusters
2. Fetch cluster credentials
3. Execute arbitrary Kubernetes commands

Using Kubernetes' IAM Workload Identity Federation, attackers can transition from the GCP realm to Kubernetes, facilitating lateral movement. This access enables malicious actors to:

1. Extract container images using tools like crictl
2. View and export large language models (LLMs) and fine-tuned adapters

Risks of Malicious Model Deployment

The research highlights that deploying even a single trojanized model on a production system could compromise an entire AI environment. Attackers could use such models to exfiltrate sensitive data, including proprietary ML models and fine-tuned LLMs.

Mitigation Measures

Following responsible disclosure, Google has patched both vulnerabilities. Organizations using Vertex AI are advised to:

1. Enforce strict controls on model deployments
2. Audit permissions for deploying models in tenant projects
3. Regularly review and restrict custom job pipeline access

Broader Implications in AI Security

The findings come alongside revelations from Mozilla's 0Day Investigative Network (0Din) about potential interactions with OpenAI's ChatGPT sandbox. Unlike the Vertex AI flaws, OpenAI considers such interactions as intended functionality within its sandbox environment.

"Activities like uploading files, running scripts, or executing commands are features of the sandbox and do not pose external security risks," said Marco Figueroa, a security researcher.

Final Thoughts

The vulnerabilities in Vertex AI underscore the importance of securing ML environments against privilege escalation and data exfiltration attacks. As AI systems become integral to business operations, safeguarding them against malicious actors is paramount.