How to Remove Client Secrets from Source Code: Part I (the code part)

I wanted to share the music4dance code with a potential employer, but I didn’t want to share all of the various keys and secrets that I use to shared login services, music search services and such.  Since I hadn’t previously shared the code with anyone other than people that I would trust to share the keys as well, I have been pretty sloppy with just checking them in with source code.  This is particularly nasty since that meant that if I actually wanted to share the git repo, I would not only have to clean up the code, but I’d have to change every one of the client secrets and passwords.

It’s pretty well documented in a number of places that a reasonable way to do this is to save such sensitive information in the environment and load it at runtime.  That way it will never get checked into the source code.  The problem that I haven’t seen a nice solution to online is dealing with a whole bunch of these at the same time.  There are two parts to this, the changing of the source code and setting up the environment on all the machines that need it.

The documented way to load an environment variable is straightforward:

public string MyKey => Environment.GetEnvironmentVariable("mykey");

And if there’s some possibility that the property is read multiple times (and generally there is), you can easily cache the variable (I love some of these new C# syntax enhancements).

public string MyKey => _myKey ?? 
    (_myKey = Environment.GetEnvironmentVariable("mykey"));
Private string _myKey;

Now, I’m doing this in a dozen or so places in my code, and almost all of them conform to a pattern where I am storing a pair of values.  Either client key and secret or username and password.  So I threw together a little class to handle this.

public abstract class CoreAuthentication
{
    protected abstract string Client { get; }
    public string ClientId => _clientId ?? 
        (_clientId = Environment.GetEnvironmentVariable(Client + "-client-id"));
    public string ClientSecret => _clientSecret ?? 
        (_clientSecret = Environment.GetEnvironmentVariable(Client + "-client-secret"));

    private string _clientId;
    private string _clientSecret;
}

The idea for this is that I had a number of specific authorization handlers for each of the music services that would extend this class and override the Client property to let CoreAuthentication know what environment variable to read.

Then for the simpler cases where I just needed to grab the info and use it to an OWIN security provider, I created a generic subclass that just takes the client name that I use as the base of the environment variable name.

public class EnvAuthentication : CoreAuthentication
{
    public EnvAuthentication(string client)
    {
        Client = client;
    }

    protected override string Client { get; }
}

I use the EnvAuthentication class in my startup authorization like so:

var fbenv = new EnvAuthentication("fb");
var fb = new FacebookAuthenticationOptions
{
    AppId = fbenv.ClientId, 
    AppSecret = fbenv.ClientSecret
};

Not rocket science, but a somewhat cleaner solution than having Environment.GetEnvironmentVariable calls all over my code.

I’ll attack the problem of loading up all of those environment variables next time. Especially the case of batch loading into the azure app service, which I found particularly vexing.

Restarting Azure Search

I had the misfortune of discovering that my Azure Search Service had been stopped unexpectedly.  In the end, this turned out to be my fault (I let a subscription lapse), and the azure search folks were a great help in resolving the issue.  But the bottom line was, when I saw that the website had broken and tracked it down to the search service not responding, I was stuck.

Here’s what I saw:

Search with Status = Stopped

After some frantic searching around the portal interface as well as asking the internet I concluded that there wasn’t any way to restart the service from the portal.  What didn’t occur to me since I’ve never used PowerShell for azure management in the past is that there is a reasonably easy way to start and stop the search service (and presumably other services) from PowerShell.

Here’s what I did:

Step 1: Install the Azure PowerShell cmdlets

I followed the instructions here and used the second method – Installing from the WebPI.

Step 2: Logging in

Since I have multiple subscriptions, I couldn’t just use the standard login, I had to add the SubscriptionId parameter.

PS >Login-AzureRmAccount -SubscriptionId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Environment           : AzureCloud
Account               : xxxxx@music4dance.net
TenantId              : yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy
SubscriptionId        : 
SubscriptionName      : BizSpark
CurrentStorageAccount : 

Step 3: Start the Search Service

PS >Invoke-AzureRmResourceAction -ResourceType "Microsoft.Search/searchServices" -ResourceGroupName "My-Resource-Group-Name" -ResourceName "my-resource-name" -ApiVersion 2015-08-19 -Action "Start"

id         : /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroupsMy-Resource-Group-Name/providers/Microsoft.Search/searchServices/my-resource-name
name       : m4d
type       : Microsoft.Search/searchServices
location   : West US
properties : @{replicaCount=1; partitionCount=1; status=running; statusDetails=; provisioningState=succeeded; hostingMode=Default}
sku        : @{name=free}

And Voila, the search service is restarted and the website is back up and running again.

Search with Status = Running

Step 0: Errata

Of course, this didn’t go as smoothly as I outlined. There were two major blockers. First, I didn’t think to pop down to the PowerShell tools when the portal didn’t do what I needed it to do. That’s a lesson that I’m hoping I only have to learn once.

The second issue was that I didn’t actually do a clean install of the PowerShell extensions, assuming that since I was keeping Visual Studio up to date including Azure extensions that would take care of PowerShell. So my first attempt to start the service looked more like this:

PS >Invoke-AzureRmResourceAction -ResourceType "Microsoft.Search/searchServices" -ResourceGroupName "My-Resource-Group-Name" -ResourceName "my-resource-name" -ApiVersion 2015-08-19 -Action "Start"

Invoke-AzureRmResourceAction : Run Login-AzureRmAccount to login.
At line:1 char:1
+ Invoke-AzureRmResourceAction -ResourceType "Microsoft.Search/searchSe ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  + CategoryInfo : InvalidOperation: (:) [], PSInvalidOperationException

But I had already logged in, and apparently successfully. So WTF? After scratching my head a bit, I started looking at version information and discovered that I was on version 1 of the Azure PowerShell cmdlet, while the current version was 3.1.0. Oops.

Here’s how to check your cmdlet versions:

PS >(Get-Module -ListAvailable | Where-Object{ $_.Name -eq 'Azure' }) `
| Select Version, Name, Author, PowerShellVersion  | Format-List;


Version           : 3.1.0
Name              : Azure
Author            : Microsoft Corporation
PowerShellVersion : 3.0

And while I’m at it, here’s how to check your PowerShell version and some of the related stack:

PS >$PSVersionTable

Name                           Value                                                                                                                                                                                                                                               
----                           -----                                                                                                                                                                                                                                               
PSVersion                      5.1.14393.576                                                                                                                                                                                                                                       
PSEdition                      Desktop                                                                                                                                                                                                                                             
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                                                                                             
BuildVersion                   10.0.14393.576                                                                                                                                                                                                                                      
CLRVersion                     4.0.30319.42000                                                                                                                                                                                                                                     
WSManStackVersion              3.0                                                                                                                                                                                                                                                 
PSRemotingProtocolVersion      2.3                                                                                                                                                                                                                                                 
SerializationVersion           1.1.0.1 

Conclusion

I will be digging into more PowerShell management options since my naive assumption that the portal would give me what I needed to manage Azure Services was proven so totally wrong in this case.