Microsoft Orleans-Easily switching between "development" and "production" configurations.

There are some differences in how one would host an Orleans server locally vs deployed in a production scenario. In this post, we’ll use some of the skills we’ve built on previously, to help us in our journey to hosting a production ready Orleans cluster.

Microsoft Orleans — Easily switching between “development” and “production” configurations.

Some of the concepts we’ll be working with in this post include: appsettings, IOptions, builder pattern, extension methods, and Orleans itself.

What sort of differences will we be dealing with?

We’ve covered many of the different “options” when it comes to a local vs prod ready configuration, but never how to handle those differences. Some differences you are likely to encounter, depending on which features you’re using from Orleans include:

• The cluster configuration — this one would seemingly be encountered no matter what other feature you’re using, so we’ll be covering it in this post. Options for cluster configuration include (but are not necessarily limited to) Localhost, Azure, Db.
• Grain State storage
• Reminder services
• Probably others (but these are the ones that have had features covered thus far in posts.)

We’ve seen how Orleans makes heavy use of Builders- a Creational Pattern, which we will leverage along with a few other posts I’ve written about previously, to set up our application for multiple environments.

We’ll make use of some of the things learned in:

.net core console application IOptions configuration

to get our Orleans application updated for running under multiple configurations.

Current local configuration

For our local configuration, we are having the IClientBuilder and ISiloHostBuilder construct our cluster and clients using LocalHostClustering. That looks like:

Client:

IClusterClient client;
client = new ClientBuilder()
 .UseLocalhostClustering() // <--- this guy
 .Configure<ClusterOptions>(options =>
 {
  options.ClusterId = "dev";
  options.ServiceId = "HelloWorldApp";
 })
 .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(IGrainInterfaceMarker).Assembly).WithReferences())
 .Build();

await client.Connect(RetryFilter);
Console.WriteLine("Client successfully connect to silo host");
return client;

SiloHost:

var builder = new SiloHostBuilder()
 .UseLocalhostClustering() // <--- this guy
 .Configure<ClusterOptions>(options =>
 {
  options.ClusterId = "dev";
  options.ServiceId = "HelloWorldApp";
 })
 .Configure<EndpointOptions>(options => options.AdvertisedIPAddress = IPAddress.Loopback)
 .AddMemoryGrainStorage(Constants.OrleansMemoryProvider)
 .ConfigureApplicationParts(parts =>
 {
  parts.AddApplicationPart(typeof(IGrainMarker).Assembly).WithReferences();
 })
 .ConfigureServices(DependencyInjectionHelper.IocContainerRegistration)
 .UseDashboard(options => { })
 .UseInMemoryReminderService()
 .ConfigureLogging(logging => logging.AddConsole());

var host = builder.Build();
await host.StartAsync();
return host;

For local testing, UseLocalhostClustering is sufficient. However, it is not something we’d use in a production scenario.

What can we do to fix that? A few steps:

• Introduce configuration files into our application. These application files will contain configuration differences between our running environments.
• Configuration classes making use of IOptions.
• Changing the building of our client and silo, depending on the environment used.

Updating our solution to prepare for utilizing configuration in Orleans

We’ll be applying a lot of the same ideas from .net core console application IOptions configuration, so I may be skipping over some specifics, maybe.

Configuration POCOs Project

First, we’ll introduce a new common project that will house the POCOs and bootstrapping of our application.

Run dotnet new classlib -n Kritner.OrleansGettingStarted.Common to create the new project:

Introduce Configuration POCO

We’ll need a POCO to represent our Orleans configuration, depending on the environment. The few things that we’ll need to keep track of (at least in the way I plan on proceeding) includes:

• Silo IP address(es)
• Silo port (the port for silo to silo communication)
• Gateway port (the port for client to silo communication)

Note that much of this information is based on the documentation at https://dotnet.github.io/orleans/Documentation/clusters_and_clients/configuration_guide/typical_configurations.html. We could also apply this same logic for using Azure clustering, but I don’t have any Azure credits, so going with this for demonstration purposes.

Our configuration POCO:

/// <summary>
/// Contains properties utilized for configuration Orleans
/// Clients and Cluster Nodes.
/// </summary>
public class OrleansConfig
{
 /// <summary>
 /// The IP addresses that will be utilized in the cluster.
 /// First IP address is the primary.
 /// </summary>
 public string[] NodeIpAddresses { get; set; }
 /// <summary>
 /// The port used for Client to Server communication.
 /// </summary>
 public int GatewayPort { get; set; }
 /// <summary>
 /// The port for Silo to Silo communication
 /// </summary>
 public int SiloPort { get; set; }
}

Update Console applications to utilize configuration files

In our new class library, we’ll create a few classes to help “bootstrap” our two console apps, for pulling in configuration files, and setting up the configuration (in this case IOptions<OrleansConfig>).

Let’s add a few new configuration files:

appsettings.json

{
 "OrleansConfig" : {
  "NodeIpAddresses" : ["192.168.1.105", "192.168.1.106"],
  "GatewayPort" : 30000,
  "SiloPort" : 11111
 }
}

I’ll be using the above as the production configuration. The above configuration specifies there are two nodes (SiloHosts) to our cluster, the primary will be the first in the array, the secondary the second (you can have more than two). With the clustering configuration we’ll be using, a primary needs to be specified; with other configuration types like azure table storage, there is no “primary” node, so generally more Highly available (HA).

appsettings.dev.json

{
 "OrleansConfig" : {
  "NodeIpAddresses" : ["0.0.0.0"],
  "GatewayPort" : -1,
  "SiloPort" : -1
 }
}

In the above config, which we’ll be using for our dev environment (local testing), we’re not really giving “valid” configuration values, as these same properties aren’t used at all for LocalhostClustering. The reason I added these values was just mostly to show off the loading of a specific environment configuration file.

I’ve set up the configuration files so that they could be used for BOTH the client and silo-host, as such we’ll only want to keep a single copy of each around, so I placed them in /src/_appsettings/*.

In order to get both projects to “use” these files, we’ll link them in our csproj. The linking can be accomplished by adding the following to both the client and silo-host project’s csproj files:

<ItemGroup>
 <Content Include=".._appsettingsappsettings.json" Link="appsettings.json" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
 <Content Include=".._appsettingsappsettings.dev.json" Link="appsettings.dev.json" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>

Next we’ll need a way to load configuration files, and set up our IOptions. The following works, but I dunno if it’s the right way to do it — let me know!

ConsoleAppConfigurator:

public static class ConsoleAppConfigurator
{
 public static IServiceProvider BootstrapApp()
 {
  var environment = GetEnvironment();
  var hostingEnvironment = GetHostingEnvironment(environment);
  var configurationBuilder = CreateConfigurationBuilder(environment);

  var startup = new Startup(hostingEnvironment, configurationBuilder);
  IServiceCollection serviceCollection = new ServiceCollection();
  startup.ConfigureServices(serviceCollection);

  return serviceCollection.BuildServiceProvider();
 }

 private static string GetEnvironment()
 {
  var environmentName = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT").ToLower();
  if (string.IsNullOrEmpty(environmentName))
  {
   return "Production";
  }

  return environmentName;
 }

 private static IHostingEnvironment GetHostingEnvironment(string environmentName)
 {
  return new HostingEnvironment
  {
   EnvironmentName = environmentName,
   ApplicationName = AppDomain.CurrentDomain.FriendlyName,
   ContentRootPath = AppDomain.CurrentDomain.BaseDirectory,
   ContentRootFileProvider = new PhysicalFileProvider(AppDomain.CurrentDomain.BaseDirectory)
  };
 }

 private static IConfigurationBuilder CreateConfigurationBuilder(string environmentName)
 {
  var config = new ConfigurationBuilder()
   .SetBasePath(AppDomain.CurrentDomain.BaseDirectory)
   .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
   .AddJsonFile($"appsettings.{environmentName}.json", optional: true, reloadOnChange: true)
   .AddEnvironmentVariables();

  return config;
 }
}

Startup:

public class Startup
{
 public IHostingEnvironment HostingEnvironment { get; }
 public IConfiguration Configuration { get; }

 public Startup(
  IHostingEnvironment hostingEnvironment, 
  IConfigurationBuilder configurationBuilder
 )
 {
  HostingEnvironment = hostingEnvironment;
  Configuration = configurationBuilder.Build();
 }

 public void ConfigureServices(IServiceCollection serviceCollection)
 {
  serviceCollection.AddOptions();
  serviceCollection.Configure<OrleansConfig>(Configuration.GetSection(nameof(OrleansConfig)));
 }
}

At this point we can test our configuration file is being loaded successfully, as well as hydrating our OlreansConfig class.

Within one of the Program.Mains we can add the following, just to confirm our object is being populated (confirm in o.)

static int Main(string[] args)
{
 ServiceProvider = ConsoleAppConfigurator.BootstrapApp();

 var orleansConfig = ServiceProvider.GetService<IOptions<OrleansConfig>>();
 var o = orleansConfig.Value;
 return RunMainAsync().Result;
}

Updating the Client and SiloHost

This ended up being a lot longer than I intended… but there’s only a bit left to go, I swear!

As stated earlier, we want to use LocalhostClustering for one environment (Dev) and Develop/Static clustering for the other environment (Production).

Because we’re actually using two separate methods to configure our clustering, we’ll introduce an extension method to configure the cluster differently, based on environment. The whole reason I’m going that route is to avoid the pollution of branching logic within the client/silohost builders.

Here are those extension methods, they’re pretty straight forward…

IClientBuilderExtensions:

public static class IClientBuilderExtensions
{
 /// <summary>
 /// Configures clustering for the Orleans Client based on
 /// the Orleans environment.
 /// </summary>
 /// <param name="builder">The client builder.</param>
 /// <param name="orleansConfigOptions">The Orleans configuration options.</param>
 /// <param name="environmentName">The environment.</param>
 public static IClientBuilder ConfigureClustering(
  this IClientBuilder builder, 
  IOptions<OrleansConfig> orleansConfigOptions, 
  string environmentName
 )
 {
  if (builder == null)
  {
   throw new ArgumentNullException(nameof(builder));
  }
  if (orleansConfigOptions.Value == default(OrleansConfig))
  {
   throw new ArgumentException(nameof(orleansConfigOptions));
  }

  switch (environmentName.ToLower())
  {
   case "dev":
    builder.UseLocalhostClustering();
    break;
   default:
    var orleansConfig = orleansConfigOptions.Value;
    List<IPEndPoint> nodes = new List<IPEndPoint>();
    foreach (var node in orleansConfig.NodeIpAddresses)
    {
     nodes.Add(new IPEndPoint(IPAddress.Parse(node), orleansConfig.GatewayPort));
    }
    builder.UseStaticClustering(nodes.ToArray());
    break;
  }

  return builder;
 }
}

ISiloHostBuilderExtensions:

public static class ISiloHostBuilderExtensions
{
 /// <summary>
 /// Configures clustering for the Orleans Silo Host based on
 /// the Orleans environment.
 /// </summary>
 /// <param name="builder">The silo host builder.</param>
 /// <param name="orleansConfigOptions">The Orleans configuration options.</param>
 /// <param name="environmentName">The environment.</param>
 public static ISiloHostBuilder ConfigureClustering(
  this ISiloHostBuilder builder, 
  IOptions<OrleansConfig> orleansConfigOptions, 
  string environmentName
 )
 {
  if (builder == null)
  {
   throw new ArgumentNullException(nameof(builder));
  }
  if (orleansConfigOptions.Value == default(OrleansConfig))
  {
   throw new ArgumentException(nameof(orleansConfigOptions));
  }

  switch (environmentName.ToLower())
  {
   case "dev":
    builder.UseLocalhostClustering();
    break;
   default:
    var orleansConfig = orleansConfigOptions.Value;
    builder.UseDevelopmentClustering(
     new IPEndPoint(
      IPAddress.Parse(orleansConfig.NodeIpAddresses[0]),
      orleansConfig.SiloPort
     )
    );
   break;
  }

  return builder;
 }
}

In the above, for our “dev” environment, we’re simply using UseLocalHostClustering, and for all other environments (assuming you have more than just “dev” and “prod”), we’ll be using the configuration values as specified by the actual Orleans config. In many cases companies will have environments like “test”, “qa”, “uat”, etc.. Using separate appsettings.{env}.json allows for separate configurations, without having to make use of the old web.config transforms, or “remembering to copy the appropriate config file”. Going this route, you simply need to have the correct environment variable configured on the machine hosting the code.

Now to test, we’ll introduce some new running profiles like so…

launchsettings.json:

{
 "profiles": {
  "Dev": {
   "commandName": "Project",
   "environmentVariables": {
    "ASPNETCORE_ENVIRONMENT": "Dev"
   }
  },
  "Production": {
   "commandName": "Project",
   "environmentVariables": {
    "ASPNETCORE_ENVIRONMENT": "Production"
   }
  }
 }
}

We’ll need this in both our Client and SiloHost projects, as those are our runnable projects. You can also pass in the environment variables when running via console through commands like set ASPNETCORE_ENVIRONMENT=dev as an example.

Next we’ll update our Client build (and silo builder) to use the new extension methods

The original ClientBuilder:

client = new ClientBuilder()
 .UseLocalhostClustering() // Going to be removing this, and replacing with our new extension method
 .Configure<ClusterOptions>(options =>
 {
  options.ClusterId = "dev";
  options.ServiceId = "HelloWorldApp";
 })
 .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(IGrainInterfaceMarker).Assembly).WithReferences())
 // I don't want the chatter of logging from the client for now.
 //.ConfigureLogging(logging => logging.AddConsole())
 .Build();

becomes…

client = new ClientBuilder()
 .ConfigureClustering(
  ServiceProvider.GetService<IOptions<OrleansConfig>>(), 
  Startup.HostingEnvironment.EnvironmentName
 )
 .Configure<ClusterOptions>(options =>
 {
  options.ClusterId = "dev";
  options.ServiceId = "HelloWorldApp";
 })
 .ConfigureApplicationParts(parts => parts.AddApplicationPart(typeof(IGrainInterfaceMarker).Assembly).WithReferences())
 // I don't want the chatter of logging from the client for now.
 //.ConfigureLogging(logging => logging.AddConsole())
 .Build();

In the above, we swapped out UseLocalHostClustering for our new ConfigureClustering extension method. This method takes in our parsed IOptions<OrleansConfig> as well as the environment name. A similar change is done to the SiloHost builder.

Testing it out

Now we can use our separate run profiles to demonstrate the fact that our new builder extension method is being hit, and the client/silohost is being configured differently depending on the environment.

When running as the “dev” config:

In the above you can see our extension method is properly switching on the dev environment, and continuing to use the UseLocalhostClustering.

And running the client again, this time as Production:

In the above, you can see that our production logic clustering is being hit, and that the orleansConfig object has our correctly hydrated object.

Just for craps and laughs, let’s dotnet run the silohost and client, just to make sure everything’s still working:

There it is!

In this post hopefully we learned a little bit more about using appsettings, IOptions, the builder pattern, extension methods, and Orleans configuration.

All the code from this post (and previous posts related to Orleans) can be found:

Kritner-Blogs/OrleansGettingStarted

Related

• Getting Started with Microsoft Orleans
• Microsoft Orleans — Reusing Grains and Grain State
• Using polymorphism to update Orleans Project to be ready for new Orleans Examples!
• Microsoft Orleans — Reporting Dashboard
• Microsoft Orleans — Code Generation Issue
• Microsoft Orleans — Dependency Injection
• Microsoft Orleans — Reminders and Grains Calling Grains
• .net core console application IOptions configuration
• Code as of post — https://github.com/Kritner-Blogs/OrleansGettingStarted/releases/tag/v.50