How to Remove Client Secrets from Source Code: Part II (the environment part)

Or:  How to batch update environment variables in Azure App Services.

It’s so easy, yet so wrong to put client secrets and passwords to external services in your source code.  But for a small site like music4dance, it just didn’t seem to be worth the trouble to do anything else when the only people that had access to the source code were also trusted enough to have access to the passwords.  I talked about why I changed my mind in my last post as well as how I refactored my code to make it reasonably easy to make the changes.

The other part of the problem is that I now have a couple of dozen environment variables that I want to set on a number of machines, both locally and in the cloud and I don’t want to have to update them manually.  I can easily write a command or PowerShell script to handle local machines, but how do I manage the Azure App Services in the cloud?  It seems like PowerShell remote management would be a reasonable way to approach this, similar to the solution to restarting Azure Search, but I was completely stumped on how to to do that.

Azure Resource Explorer

Until I found this new tool called Azure Resource Explorer.  Check out David Ebbo’s video describing it here:

Azure Resource Explorer

It’s incredibly straightforward.  Just open up the website https://resources.azure.com and sign in with the same credentials that you use for the Azure portal.  There is a drop-down control in the top center where you choose your directory, make sure you set it to the directory that you’re interested in modifying if you have multiple subscriptions set up.  Then take a gander at the tree control in the left pane.

I see two top-level nodes: Providers and Subscriptions. Spend some time browsing through this tree as there is plenty of gold here.

In order to change the environment variables for my web site, I’m looking for the settings for the App Service called music4dance which is housed in the Default-Web-WestUS resource group.  So I navigate through the hierarchy like this:  subscriptions->bizspark->resourceGroups->Default-Web-WestUS->providers->Microsoft.Web->sites->m4d->config->appsettings.

Once I’ve selected app settings in the tree, I can look at the main panel, where the default tab is labeled Data (GET, PUT).  The text box has a JSON formatted object that describes the node in the tree and contains a property set of the environment variables for app settings.  We can almost stop here if we want to.  The first time through this I just copied my block of environment variables into the properties section and hit the POST button.  I went back to the azure portal, navigated to the Application Settings panel and scrolled down to the app settings panel, and there I saw my new environment variables.

Azure Resource Explorer
Azure Resource Explorer

PowerShell

But the best part is the PowerShell tab. This gives the template to view and modify the object that we’re viewing.

# PowerShell equivalent script

# List appsettings
$resource = Invoke-AzureRmResourceAction -ResourceGroupName My-Resource-Group-Name -ResourceType Microsoft.Web/sites/config -ResourceName my-resource-name/appsettings -Action list -ApiVersion 2015-08-01 -Force
$resource.Properties

# SET list
$PropertiesObject = @{
	#Property = value;
}
New-AzureRmResource -PropertyObject $PropertiesObject -ResourceGroupName My-Resource-Group-Name -ResourceType Microsoft.Web/sites/config -ResourceName my-resource-name/appsettings -ApiVersion 2015-08-01 -Force

You can just copy this into a .ps file and add in your environment variables to the PropertyObject. Don’t forget to add all the other environment variables (you can grab these in an appropriate format from the get/post tab of the resource explorer). To match up with the code that I wrote to load the environment variables, my version of SET list looks something like this:

# SET list
$PropertiesObject = @{
    #Property = value;
    "xbox-client-id" = "my-xbox-clientid";
    "xbox-client-secret" = "my-xbox-client-secret";
    "spot-client-id", "my-spotify-clientid";
    "spot-client-secret", "my-spotify-client-secret";
    # and many more
}

Putting it all together

The way I run this script is to load it into the PowerShell ISE, log into the correct subscription (see the instruction to do that here) and run the script. Pretty simple, and not prone to typos or copy/paste errors the way doing the same thing in the azure portal would be.

I think the ultimate solution would be to write a pair of PowerShell scripts, one to do the local load and one to do the remote load, both taking the same tab separated or JSON formatted file of id/secret pairs. That’s more effort than I would put in for myself, but if others are interested I’ll see what I can throw together.

How to Remove Client Secrets from Source Code: Part I (the code part)

I wanted to share the music4dance code with a potential employer, but I didn’t want to share all of the various keys and secrets that I use to shared login services, music search services and such.  Since I hadn’t previously shared the code with anyone other than people that I would trust to share the keys as well, I have been pretty sloppy with just checking them in with source code.  This is particularly nasty since that meant that if I actually wanted to share the git repo, I would not only have to clean up the code, but I’d have to change every one of the client secrets and passwords.

It’s pretty well documented in a number of places that a reasonable way to do this is to save such sensitive information in the environment and load it at runtime.  That way it will never get checked into the source code.  The problem that I haven’t seen a nice solution to online is dealing with a whole bunch of these at the same time.  There are two parts to this, the changing of the source code and setting up the environment on all the machines that need it.

The documented way to load an environment variable is straightforward:

public string MyKey => Environment.GetEnvironmentVariable("mykey");

And if there’s some possibility that the property is read multiple times (and generally there is), you can easily cache the variable (I love some of these new C# syntax enhancements).

public string MyKey => _myKey ?? 
    (_myKey = Environment.GetEnvironmentVariable("mykey"));
Private string _myKey;

Now, I’m doing this in a dozen or so places in my code, and almost all of them conform to a pattern where I am storing a pair of values.  Either client key and secret or username and password.  So I threw together a little class to handle this.

public abstract class CoreAuthentication
{
    protected abstract string Client { get; }
    public string ClientId => _clientId ?? 
        (_clientId = Environment.GetEnvironmentVariable(Client + "-client-id"));
    public string ClientSecret => _clientSecret ?? 
        (_clientSecret = Environment.GetEnvironmentVariable(Client + "-client-secret"));

    private string _clientId;
    private string _clientSecret;
}

The idea for this is that I had a number of specific authorization handlers for each of the music services that would extend this class and override the Client property to let CoreAuthentication know what environment variable to read.

Then for the simpler cases where I just needed to grab the info and use it to an OWIN security provider, I created a generic subclass that just takes the client name that I use as the base of the environment variable name.

public class EnvAuthentication : CoreAuthentication
{
    public EnvAuthentication(string client)
    {
        Client = client;
    }

    protected override string Client { get; }
}

I use the EnvAuthentication class in my startup authorization like so:

var fbenv = new EnvAuthentication("fb");
var fb = new FacebookAuthenticationOptions
{
    AppId = fbenv.ClientId, 
    AppSecret = fbenv.ClientSecret
};

Not rocket science, but a somewhat cleaner solution than having Environment.GetEnvironmentVariable calls all over my code.

I’ll attack the problem of loading up all of those environment variables next time. Especially the case of batch loading into the azure app service, which I found particularly vexing.