Preventing fallout from your CI/CD platform being hacked

Preventing cloud takeover after the compromise of credentials

Continuous integration and continuous delivery/deployment (CI/CD) tools are no longer a luxury for any startup. The fastest-moving startups have learned that shipping big, ambitious ideas works best by shipping tiny, incremental and easy-to-review changes. The most productive among them ship 40 times a day. Some even up to 80 times per day. This can only be done safely by leveraging a CI/CD tool such as CircleCI, GitHub Actions, and GitLab’s pipelines to name a few.

CI/CD attracts hackers

A lot of startups and bigger companies are using these tools nowadays. In order to have them deploy code to your cloud, you have to store special API secrets inside them. That makes CI/CD tools high-value targets for hackers. In fact, they have a history of being hacked all the time.

Have a look at these incidents, which are just some of the recent breaches that have been publicly disclosed:

CircleCI breach retrospective w/ IOCs and TTPs

Quick TLDR:

1. Malware on eng laptop
2. Stole active SSO session for a remote session
4. Generated production access tokens
5. Exfil'd customer ENVs, tokens, keys.
6. CircleCI encryption keys exfil'd too.https://t.co/25x9t5NLG6

— Ryan McGeehan (@Magoo) January 14, 2023

CircleCI got breached in January 2023

• CodeShip: “Critical Security Notification: GitHub breach” (2020)
• GitHub: “Exploiting GitHub Actions on open source projects” (2022)
• GitLab: “Action we've taken in response to a potential Okta breach” (2022)
• Jenkins: “A critical Jenkins bug discovered” (2020)

As you can see, it happens pretty regularly. How are you defending yours?

How do I defend my cloud infrastructure against breaches like these?

When one of these CI/CD platforms gets hacked, they will usually disclose the breach. That tends to happen within a day of them noticing the breach. However, a breach could be active for weeks before they find out. Unfortunately, that time can be used to escalate access to all customers of the platforms.

Aikido helps you identify your CI/CD defenses

Luckily, there are some methods to make sure you stay safe even if your platform of choice gets hacked. Aikido Security’s new integration with AWS will alert you if your cloud does not actively take any of the following measures. Use our free trial account to see if your cloud already has defenses against these threads.

Steps to take to defend your CI/CD:

1. When assigning IAM roles to your CI/CD platform, make sure they are restricted by IP. Most CI/CD tools have an option to only send traffic from a specific set of IP addresses. That option renders stolen API tokens unusable outside of the CI/CD infrastructure. A hacker won’t be able to use them on their own servers, which should slow them down a lot and potentially block them altogether.
2. When creating credentials for CI/CD platforms, spend time crafting minimal access. Don’t give out admin rights.
3. Don’t put all your eggs in one basket: split your clouds up into multiple accounts. This minimizes the effect of a breach. For example, a breach of your staging environment credentials should not result in a breach of your production environment.
4. Use single sign-on (SSO) or multi-factor authentication (MFA). A no-brainer really.

Sadly (but realistically), you should assume your CI/CD will get hacked one day. So when the time comes, make sure to rotate all deployment tokens asap.