AWS Bedrock Enumeration via ValidationException

Goal

Detects potential enumeration of AWS Bedrock models through a spike in failed InvokeModel calls.

Strategy

This rule monitors CloudTrail for failed InvokeModel API requests to bedrock.amazonaws.com, where the error code is ValidationException and the error message references max_tokens_to_sample. Repeated failures of this kind resemble automated tools probing model token limits and parameter boundaries. This is a reconnaissance technique used to enumerate available models and their input constraints ahead of abuse or data exfiltration. This error is also possibly related to misconfigured applications, especially when it’s high-volume and repetitive from a known principal.

Triage and Response

  • Determine whether {{@userIdentity.accessKeyId}} maps to a trusted application or user, and whether Bedrock access is expected for that principal.
  • Review the full set of InvokeModel calls from this access key to gauge the range of models probed and the volume of errors generated.
  • Check for signs that this key is compromised, such as recent key rotation or calls originating from unexpected IPs or regions.
  • Identify any successful InvokeModel calls within the same time window as the errors, which may indicate the enumeration was partially successful.
  • Pivot to other CloudTrail activity from this principal for potentially related discovery behaviors, such as ListFoundationModels or GetFoundationModel calls.