[colug-432] Puppet Foreman default environment

[Scott Merrill]

> > Additional classes can be applied at various other levels of our Hiera hierarchy, though mostly they’re set on a per-server level.
>
> If I didn't mention this before - thank you for this explanation.  This has been a massive help and significantly shifted my thinking about how to manage the applied classes and class parameters. 

You're welcome!  Hiera and the data separation it provides is not an obvious "win" at first glance. It looks like one more layer of _stuff_ to manage and maintain; but the long-term gains are definitely worth it.

> I came back to this email to ask a question about managing what could easily be an out of control hiera.yaml hierarchy - but you answered the question already.
>
> For now, I've landed on
> - node/%{::fqdn}
> - datacenter/{%::datacenter} # based on a custom fact
> - common # what you called base
>
> I think your comment about "mostly set on a per-server level" is what would keep the hierarchy from spinning wildly out of control.

The Puppet Labs folks have a (possibly undocumented) rule of thumb that you should keep your hierarchy to about 5 or 6 levels. Much more than that and you struggle to find the data you want scattered through dozens of small files; much less than that and you struggle to find the data you want inside a few enormous files.

The server-based YAML configs are good when the number of servers is relatively small. We've found that a rapidly increasing server count brought with it a rapidly increasing YAML count. In our context, this worked very strongly *against* our automation goals. We could spin up a server in no time, but manually crafting, committing and deploying the server's YAML took a lot of time and effort (relative to the task of creating a new VM).

We've been working to make Puppet a little more reactive on some of our systems based on values of custom facts that interrogate the system. So rather than saying in advance "server123 is an app server for application foobar" we say "server123 is an app server because it has a custom fact declaring this to be the case." Then additional custom facts identify that the code for application foobar is present so Puppet does the necessary stuff.

In this way, we can Kickstart a generic app server and let Puppet do the necessary actions after some app code has been deployed, rather than crafting a server-specific YAML file.

Its a work in progress, but things are looking good so far.

Cheers,
Scott