In November 2020, an outage of Amazon Web Services (AWS) in the us-east-1 region rendered Roomba vacuum …
In this blog post, we dive into a comparison of CodeShield versus other existing open-source tools.
IAM vulnerable is a GitHub project consisting of 31 examples of IAM privilege escalations paths that can be deployed using Terraform. Once deployed, the project allows you to learn about all different types of IAM privilege escalations, or to benchmark different tools that detect those escalations. We definitely recommend trying it out yourself – in a sandbox account.
Using IAM vulnerable, Seth Art performed an in-depth comparison of a set of open-source tools (PMapper, AWSPX, Cloudsplaining, and Pacu) and wrote a blog post. We use Seth Art’s “Capability Summary” table and extended it with CodeShield. (See Table below). The capabilities are all IAM specific features.
|Capabilities||PMapper v1.1.3||AWSPX v1.3.4||Cloudsplaining v0.4.5||Pacu v1.1.4||CodeShield v2.0.30|
|Reports on user privsec?||Yes||Yes||Yes||Yes||Yes|
|Reports on role privesc?||Yes||Yes||Yes||Yes||Yes|
|Handles resource constraints?||Yes||Yes||No||No||Yes|
|Considers service roles?||Yes||Yes||No||No||Yes|
|Handles transitive privesc paths?||Yes||Yes||No||No||No|
|Handles DENY actions in a single policy?||Yes||No||Yes||Yes||Yes|
|Handles DENY actions in multiple policies?||Yes||No||No||Yes||Yes|
|Handles condition constraints?||No||No||No||No||Partially|
|Handles service control policies?||Yes||No||No||No||No|
For instance, the feature “Reports on user privsec” and the feature “Reports on role privsec” are important to understand when judging the output of the results. In the case a found privilege escalation starts at an IAM role, one must further understand whether the role is assumable and by whom. A role can be assumed by AWS SSO, by a user, but also by any other resource. For all details of the features in the table, we refer to the original blog post.
Using IAM vulnerable we also did benchmark CodeShield. We deployed IAM vulnerable to a sandbox account and scanned it using CodeShield. We did not find a single false positive, i.e., there is no noise in using CodeShield. As of today, 6 of the 31 scenarios are still missing. Those will be modeled and shipped with the next release, the same holds for other missing features, such as service control policies.
CodeShield vs. PMapper
As you see, the most related tool to CodeShield is PMapper.
Apart from modeling the policy evaluation correctly and precisely, we also made sure to keep up performance. For instance, on large AWS accounts, we realized PMapper doesn’t have the best performance – it sometimes takes about 3-12 hours to complete. This makes it extremely difficult to use as you quickly need to launch it on a dedicated cloud machine.
Yet another challenge is to keep up with all sorts of IAM privilege escalations, being continuously up to date with the most recent attack vectors on IAM, and maintaining the policy evaluation engine really is a full-time job.
Third, understanding privilege escalations and the risks they introduce is a complicated problem that needs to be visualized clearly for a user to understand the impact and the risk each escalation has attached.
Due to these three drawbacks, we decided to implement and launch our own IAM privilege escalation detection engine as part of CodeShield. We carefully designed it in a way, that it returns results quickly (typically within 30 minutes), allows us to easily model new IAM attack scenarios and ensure that with each scenario, the tool also explains the associated risk in detail.
If you want to try out CodeShield yourself, sign-up, connect an AWS account, and perform a scan. We recommend you to deploy IAM vulnerable and convince yourself.