Skip to main content

Generic API Key Detection: Finding the "Unknown Unknowns"

The Problem with Regex​

Most security scanners rely on Regular Expressions. They look for known patterns like AKIA... (AWS) or sk-... (OpenAI).

But what about Pinecone? What about Weights & Biases? What about your company's internal API? These keys often look like random strings of 32-64 characters. There is no fixed prefix. A regex scanner will miss them entirely.

The Solution: Shannon Entropy​

To find these "unknown" secrets, we use mathematics. Specifically, Shannon Entropy.

In information theory, entropy measures the randomness of data.

  • Low Entropy: "password123" (Predictable, repeating characters).
  • High Entropy: "7f8a9d12-b3c4-9e8f" (Random, chaotic).

API keys are generated to be random. Therefore, they have high entropy.

Context-Aware Scanning​

High entropy alone isn't enough (a compressed zip file also has high entropy). We need Context.

Veritensor combines entropy analysis with variable name heuristics. It looks for code patterns like:

# Variable name indicates a secret + Value is high entropy
pinecone_api_key = "89d8f9a8-d9a8-4b8a-..."

If the variable name contains key, secret, token, or auth, AND the assigned string has high entropy, the scanner flags it.

Why This Matters for AI​

The AI ecosystem is fragmented. You use dozens of niche tools (Vector DBs, Tracing tools, GPU clouds). Most of them don't have standardized key formats. Using an entropy-based scanner is the only way to catch leaks for tools that didn't exist six months ago.