Risks in the image registry

All of these containers are images … somewhere. That ‘somewhere’ is an artifact (or container) registry. What risks can we find in our container registry?

Broadly speaking, there are two types of container registries - public registries and private registries. Public registries are websites on the open internet that anyone can publish their software too. These include a ton of popular websites like

• DockerHub , operated by Docker
• Quay.io , operated by Red Hat
• gcr.io , Google’s container registry
• ghcr.io , GitHub’s container registry
• and many more

Private registries are often self-hosted versions of the same software, although other popular vendors exist in this space too such as JFrog’s Artifactory or Sonatype Nexus .

All of these systems have a couple key risks. We’re going to emphasize the risks that happen between public and private registries and between a company’s internal system and the internal (private) registry. The same problems are also true of individuals pulling directly from a public registry, though.

Once you’re talking to a larger-scale organization, a few details change. First, they’ll typically want to keep things internal versus having everyone reach out to DockerHub (and get rate limited). To do that, things look more like this:

add in a random assortment of “safety checks” and you end up with something like this

Now security gets involved, and suddenly there’s a few more “things” in between developers using the image and the upstream source somewhere on the internet. These things vary based on the company, the sensitivity to risk, the existing tools and vendors there, and probably a lot more. Common checks include:

• scanned (some container scanner)
• scanned again (if one scanner is good, two is better)
• verified to not have been tampered with (this is where Cosign comes in)
• build attestations (not as common, but ties a build to an artifact)
• software bill of material (SBOM) downloaded (increasingly common)
• sprinkled with ✨ magic compliance pixie dust ✨
  • custom SSL certs
  • logging
  • other configuration
• … and so on …

I call this the “MirrorBot 9000”. It could have been built in-house. Nowadays, it was probably purchased from a vendor. These systems may be great, but also … they introduce their own risks. 🙈

Insecure connections to registries

The first step in all of this is making sure the traffic between your container host, registry, and whatever it’s using upstream are all encrypted. It sounds simple, but a shocking number of teams I work with haven’t set this up.

Much more common though, is that this is one more thing to check when you have your own zero trust MitM proxy. These are used for doing a “break and inspect” of encrypted communication. If anything in this path doesn’t have the correct certificates installed to allow it, the system sees this as malicious activity and connectivity fails. Rather than debugging, it’s much easier to just disable certificate validation … completely nullifying any security gains from these systems and introducing extra risk.

😈 Exploit example - This makes typosquatting attacks much easier, both upstream and inside your network boundary. Since a certificate would be issued to official-repo and not officia1-repo, it would get caught by not having the correct certificate. However, so long as the malicious one presents a certificate (even self-signed), it doesn’t matter if it’s trusted or valid - the connection is accepted and you’ll get served a potentially compromised copy of whatever image was requested.

I mean … “stop doing this” seems simplistic, but that’s genuinely the answer here. I have a fondness for searching version control systems for settings that’d indicate it’s disabled, like curl -k or --allow-insecure-registry or --tlsverify=false as a good starting point.

Stale images

This internal registry is configurable, but here’s where some danger lies.

1. Tags are mutable, meaning they can be reassigned. This is a good thing, in that latest can always be the latest, but also bad in that if you think you’re using v2.3.4 that can be reassigned to a new image.
2. The registry needs to set a time to expire cached images. As an example, if you pulled latest, that could stay forever in your system never being updated.
3. Make sure sync jobs alert on failure! Silent failures mean drift doesn’t get noticed as time passes.
4. Not updating your security software isn’t how you stay safe either … yes, for real life. 🤦🏻‍♀️

Look at your registry’s documentation and configure it to stay ✨ fresh ✨

Authn and Authz are hard

These registries each have their own settings for authentication and authorization. These are specific to what registry you chose, how many and how sensitive each project is, and so much more. The long and short of it is this:

That said, a few pointers:

• Don’t use the “built in” authentication if you have a dedicated identity provider.
• Long-lived secrets aren’t secret for long. Avoid robot or service accounts as much as possible.
• Check, and double check, then set a calendar reminder to check again how projects are set up there.

😈 Exploit example - I’m still never tired of searching for secrets in repositories, but this is something that has much more nuance. Try to map out where “production” is pulling from and who/what can write to that. There could be some room for error here.

Real world success patterns

It’s easy to say if you’re “disabling” it, you’re probably wrong.

• Scanners for vulnerabilities, malware, and other shenanigans
• Certificate verification
• Alerts for failures
• Integrity checks of the image
• … and so on …

Behind every --allow-insecure-registry is an engineering team that cares about their product. Growing tired of waiting on another team to unblock them, teams will unblock themselves.

The more challenging security risk is that people do their job first, even if something is in the way. Keeping cross-team requests documented and turned around quickly is critical to registry (and system) security at any scale.

Up next - what’s in those images anyways? (back to the summary)