It's not the lack of "advanced" constructs as you say. It is also the total disregard for all the human centuries of effort that have gone into making programming languages and environments convenient vehicles for expressing computation. Things like stack traces, syntax highlighting editors, auto-complete, interactive debuggers, and a bunch of other stuff you get with an actual programming language that you don't get with an external DSL. There is a reason those things are not part of the package. It is because making those things is extremely hard. Whereas slapping together a parser is not but feels like actual work.

You might consider those things advanced. I consider it the bare minimum to get proper work done.

Here's the thing: HCL isn't a programming language nor will it ever be. I hope. this problem isn't unique to Terraform; Chef and Puppet are in the aame boat.

It's markup. It is meant to be easily readable by devs and non-devs alike while being reasonably capable.

The beauty of these tools is that they are written and be expanded upon with popular languages. If I wanted to make Terraform deploy Dell servers, all I have to do is write a plug-in with Go. What's more is that I don't have to teach a TF user anything new since its usage pattern will be the same as any other TF plug-in.

Nope. Chef is an embedded DSL. It does not have the same shortcomings. In fact it is the only configuration management tool that approaches the problem from the right angle. Instead of re-inventing the wheel it gives me a library of composable components that I can glue together however I wish.

You must not have seen many TF deployments. I've seen several and each one is a unique snowflake. Some use Jinja, some use Makesfiles, some use Erb, some just use heroic copy-pasting. There are no conventions or standards whatsoever. I don't know what you mean by readable either. When was the last time you navigated 20 Terraform modules while mentally filling in all the variables and thought to yourself "Ya, this is totally readable".

There is also Rex that let you write Perl and use any Perl library anywhere.

> It's markup. It is meant to be easily readable by devs and non-devs alike while being reasonably capable.

Who are the non-devs reading infrastructure code? I think the target market for Terriform (probably) understands the basics of programming languages like loops, conditions, and variables.

I see your point about plugins/expansion. I think infrastructure hasn't gotten past the "write a plain file as an API" stage, so writing a library to generate those files feels like hacking on top of an incomplete platform.

With Puppet 4, Puppet is lightyears ahead of the mess that is HCL. It's not a general-purpose language, but it's strongly typed and you can work with datastructures in a functional fashion. I'll admit that some more complicated transformations are less natural than I'd like, but at least they're possible.

If I could use puppet to manage openstack instances at work, I'd use it over Terraform in a heartbeat because the foundation is much more solid, but unfortunately such a provider does not exist.

I use terraform and want to like it, but every bug and weird behaviour and nonsensical limitation is making it harder and harder. Furthermore, these aren't really fixable implementation issues. The terraform language needs a complete redesign.

You don't need explicit loops and branching to do the kind of declarative programming terraform needs, but being able to use basic datastructures besides strings would be very useful.

Terraform has lists and maps, which takes care of a number of use cases at least. Now, if you want types as opposed to data structures then you start to get into structures that are tough to serialize into JSON that is helpful for interoperability with even less featureful languages / DSLs.

Instead of if statements, pattern matching constructs would probably suffice for avoiding some of the concerns with avoiding loops and branching (limits and predictability upon the DAG and resulting state machine being generated as I understand it).

In any case, I've found plenty of power by using custom data providers that use whatever logic I want and feed that to Terraform providers and modules to instantiate. I think the level of maturity with the Terraform community is still in the early stages and emphasis will shift to data provider based constructs for orchestration.

What would you consider has the bare minimum to get the job done then? Chef? Ansible? Salt? Puppet? Some homegrown monstrosity? I have used the first two and both of them have major problems with state and they can get super complicated if you are not careful.

I think part of your issue with Terraform is that you want it to do everything, when you should just use it to get the foundation up and then use a provisioner with more power to do the finer detail work to fit your needs. Also, lets remember it's not even 1.0 software at this point. We started playing around with it around 0.6 and things have come along way since then. Maybe it's just not the tool for your needs.

You're setting up a false dichotomy. Expressing a dependency graph of cloud resources does not require a special DSL.

In fact if Terraform was a library it would not do everything nor do I want it to do everything. At the end of the day Terraform traverses a dependency graph and generates a sequence of commands to run. The traversal is basically a topological sort and the command generation is a bunch of API calls. None of those things require a specialized DSL.

And you still didn't answer my question. Based on your other comments Chef is your tool of choice for setting up servers because you can hack and glue things together how you want. That's great for you I get it. I assume you are a programmer by trade that wants to use the tools you feel comfortable with to get the job done. That's OK. I am an ops guy that is looking to make reproducible infrastructure without having to figure out awful Ruby/Chef bastardization. Terraform isn't a programming language. I understand you wish it was, but it simply isn't one. It's not the tool for someone that wants to program the infrastructure with Ruby.

I understand your point, but as a templating language the following should be expressible:

Given a list:

    [ A, B, ... N ]

Create a list of objects:

    [ 
        { foo: const, bar: A }, 
        { foo: const, bar: B }, 
        { foo: const, bar: C }, 
        ...,
        { foo: const, bar: N }
    ]

You say, why, this complicated programming task is not a job for terraform. Then why are some elements in place of the dsl functions, and some missing?

I have a feeling that the creators of may cloud tools don't use the cloud, or only to have pet infrastructures instead of pet servers. To set up something automated, complicated, scalable, reproducible, you have to result to do it yourself, while these tools are supposed to do it for you.

I feel like ops people are writing these tools for ops people, but instead of good old bash, now in Go. What is the problem with this?

* They reinvent well known patterns, but call them differently.

* They disregard the common knowledge of software development tradecraft.

* They don't get these things right for the first (or second time), just redo their previous workflows, and we are still where we were 10 years ago, just with different tools.

I personally prefer Ansible, but it has its weak points as well, and its development seems to have slowed since RedHat acquired it. The main advantages of Ansible are extensibility and reuseability. I can write custom modules, and have a descriptive DSL for stuff not thought of for the upstream devs. I can also simply reuse parts, which is a very weak point in terraform.

Still we use terraform, for it has its own merits, but IMHO if you want to do something, either do it, or don't do it, but don't do a half-assed "solution" which promises much, and fails to deliver. This is my feeling with terraform, where I could not even create a simple mapping of values, if those are not strings! (I know, "ops" people love strings. Only those pesky "devs" love structured data)