Using GCP Service Accounts to Access AWS IAM Roles

Imagine for a second that your company has compute resources and workloads on Google’s Cloud Platform (GCP), and that you also need to manage resources within an AWS account. This multi-cloud scenario is becoming more common nowadays, as organisations experiment with different service offerings – but it does present some challenges.

Not least of those challenges is how to manage credentials when systems in one cloud provider need to manage resources in the other. The easy way isn’t always the best.

SETTING THE SCENE

For the sake of clarity, let’s say that you already have an AWS account, and a GCP account. Your company has decided to deploy CI/CD systems in the GCP account, but wants to use those pipelines to deploy and manage resources in the AWS account (this isn’t an entirely fabricated example, by the way, it’s based on a true story).

In this example, the job running inside a GCP VM needs to interact with an SQS Queue to publish a message in an AWS account; an equivalent command-line might be:

aws sqs send-message --queue-url https://sqs.ap-southeast-2.amazonaws.com/123456789012/example-queue --message-body '{"message":"hello, world"}'

Pretty simple, and not uncommon. In order for this command to succeed, the CI/CD pipeline user (the identity that the build agent runs as) must have AWS credentials available to it, and those credentials must be able to make the relevant API calls.

THE SIMPLEST (AND MOST DANGEROUS) WAY: IAM USERS

Creating an IAM User in your AWS account allows you to generate an Access Key and corresponding Secret Key, export them, and inject them somehow into your build pipeline; whether via the capabilities of the CI/CD tool (eg storing them as a “secret”), putting them on the filesystem of the build agents, injecting them as environment variables, or something else.

The risk here is that these credentials never expire, they have to be transferred somehow from the AWS environment to the GCP environment, and in most cases, people want them to be stored somewhere so that they can be used to re-create the CI/CD environment later if required.

The number 1 route for AWS account compromise (and subsequent exploitation for, say, mining cryptocurrencies) is via IAM User credentials; unwittingly publishing them to a GitHub repository, deliberately putting them into a shared documentation page, storing them on a USB stick and accidentally leaving it in a taxi … the list is endless.

There are multiple clever solutions for storing IAM User credentials securely (I should know, I wrote one of them: http://credulous.io ), but all of them work around the point: if you never create these credentials, you never have to manage them.

THE MOST SECURE WAY: IAM ROLES AND IDENTITY PROVIDERS

By this point, someone reading this blog post will have piped up: “just use IAM Roles!” and they’d be right. IAM Roles offer the same capability to perform AWS actions as IAM Users, plus the credentials are only valid for a limited period of time (between 15 minutes and 12 hours, depending on the settings on the role itself). This nicely avoids much of the risk associated with long-lived credentials leaking out into the wild.

Of course, the challenge is: how do you get these time-limited credentials into a VM running inside GCP? You certainly don’t want to have to copy them in manually each time.

Let’s bring AWS IAM Identity Providers to the rescue. IAM supports creating a trusted identity provider resource, based on some external attributes, which will then be trusted by the IAM Role and can be relied upon to generate time-limited credentials via the Security Token Service.

That suddenly went fast, didn’t it? It’s ok, I’ll break it down.

IDENTITY PROVIDERS

In IAM-speak, an Identity Provider (IdP) is some external service which can be trusted to authenticate an entity. IAM takes some kind of cryptographically-signed assertion from that trusted provider, and generates time-limited credentials which are then issued to the entity. Each time the entity needs new credentials, it goes via the IdP, gets a fresh assertion, presents it, and gets fresh credentials.

IAM trusts two types of external IdP natively: ones which use a protocol called Security Assertion Markup Language (SAML), and ones which use OpenId Connect (OIDC).

GOOGLE SERVICE ACCOUNTS

When you create a VM inside GCP, you can associate a Service Account with that VM. The VM can then use automatically-generated, time-limited credentials of the service account to make GCP API calls. You can think of this as being pretty much identical to the way that IAM Roles and IAM Instance Profiles work inside AWS with EC2 instances.

It turns out that the Service Account credentials are OIDC Tokens, which can be presented to an OIDC IdP inside AWS via the sts:AssumeRoleWithWebIdentity API call to return … IAM Role credentials!

LOVELY! HOW?

CREATE SERVICE ACCOUNT IN GCP

Before you create the VM in GCP, you should create the Service Account that’ll be used. You could just use the default Service Account that GCP creates for all Compute VMs, but then any VM in your fleet could acquire IAM credentials at any time, and you probably don’t want that.

  1. In the GCP console, go to the IAM & Admin menu, then choose Service Accounts.
  2. Click “Create Service Account”
  3. Fill in the details of the service account name and its description and click Create
  4. In the Permissions screen, add the “Service Account Token Creator” Role and click Continue
  5. You do not need to grant users or groups access to this service account, and you do not need to create keys. Just click Done.

Once the Service Account is created, click on it to see its details

  1. Click “View Domain-Wide Delegation Client ID”
  2. Copy the numeric Client ID – you’ll need it for creating the IAM Identity Provider in AWS step (below)

CREATE VM IN GCP WITH ASSOCIATED SERVICE ACCOUNT

Now we’re going to create a simple VM and associate the service account that we’ve just created with it. Processes running inside that VM will then be able to act as that service account.

  1. In the GCP console, go to Compute Engine menu and then choose the VM Instances submenu
  2. Click Create to start building a new VM
  3. Name the VM, and choose an appropriate region for it; I used australia-southeast1 because that’s the closest to where I am, but it really doesn’t matter. I also chose an n1-standard-1 type, which is a small VM that doesn’t cost very much. I chose Ubuntu as the operating system.
  4. In the Identity and API Access section, click on the pulldown which (by default) says “Compute Engine default service account”; you should see the name of the service account that we created above as an option, and you should choose that service account.
  5. Click Create and wait for the VM to be active

CREATE IAM IDENTITY PROVIDER IN AWS

Using the code at https://github.com/cevoaustralia/gcp-sa-to-aws-iam-role

  • Using the numeric Client ID you found above, create the CloudFormation stack (you’ll need to be using credentials which allow you to create and manage IAM Identity Providers, Lambda Functions, and IAM Roles):

    $ CLIENT_ID= make
    

    This creates an OIDC IdP which trusts the Google Accounts signing key, which will be used to authenticate the credentials from the service account

  • Once the stack has created, copy the DeploymentUserRoleArn from the Outputs and save them for the next step.

USING THE GCP SERVICE ACCOUNT

Here’s the payoff.

  • Log in to the GCP VM which you created above (you can use the browser window, or the gcloud command-line, or … really, whatever you like)
  • Download the GitHub repo code, which contains the python code which will fetch the AWS credentials (you can use any AWS SDK, Python is just what I wrote this example in):
   wget https://github.com/cevoaustralia/gcp-sa-to-aws-iam-role/archive/master.zip
  • unzip the downloaded archive:

    unzip master.zip
    
  • change to the directory containing the example credential acquisition code:

    cd gcp-sa-to-aws-iam-role-master/gcp
    
  • Install the dependencies:

    pip3 install -r requirements.txt
    

Before running the code, it’s worth explaining what it’s doing: it requests an OIDC token from the local VM’s metadata endpoint, then calls the AWS AssumeRoleWithWebIdentity API using that token. The resulting AWS credentials are written into a standard location, at $HOME/.aws/credentials after which they can be used automatically by the AWS CLI

To acquire those AWS credentials, simply run the script with the ARN of the IAM Role that you copied from the stack outputs, above:

./get_aws_creds.py arn:aws:iam::123456789012:role/DeploymentRole

Congratulations! You now have a GCP VM with time-limited credentials for the IAM Role available to it. You can validate this via the AWS CLI tool:

aws sts get-caller-identity
{
    "UserId": "AROAYS4LBXD7LMQMP42FD:example",
    "Arn": "arn:aws:sts::123456789012:assumed-role/DeploymentRole/example",
    "Account": "123456789012"
}

THE LAST BITS

These credentials are only valid for a limited time period, of course – by default, 3600 seconds (1 hour). If you want to keep these credentials fresh, you could use a cron job, or use the above script as a pre-execution task before running your actual awscli commands … the possibilities are endless.

CONCLUSION

I hope this has been helpful to you – demonstrating how you can acquire and use AWS time-limited, role-based credentials on a Virtual Machine in Google Cloud.

If you’d like to know more about identity management; architecting, deploying, migrating or managing workloads in AWS; or multi-cloud and hybrid cloud environments, please get in contact with us; we’re the Cloud Enablement Experts!

Enjoyed this blog?

Share it with your network!

Move faster with confidence