A lot of great new features in this release - very excited to see the tool become more and more powerful.
I only came across Ansible this summer, but it really blew my mind. I've been doing systems contracting for a few years now and Ansible is the most valuable tool I've come across for systems management so far. Now, rather than have to use (and train my clients on) multiple tools, I can have them just learn one solid tool that can handle orchestration, deployment, configuration management, server inventory, etc. I can now do a systems project in days when before the same project would take weeks. Expect to hear a lot more about Ansible in the future.
Ansible isn't awesome because of a feature-list. It's awesome because it saves me time and gives me a lot of power for very little investment. Whether you're setting up a single server or 10,000 nodes, you can use this tool to give you a ton of leverage without having to spend weeks learning complex tools.
That's why I've been focusing on it so much in the last few months.
Your book looks fantastic - seriously considering getting a copy right now.
I've just started learning Ansible, and I'm interested in using it for slightly more complex setups than the typical "deploy my Django app to my VPS" scenario - e.g. EC2, load balancing, RabbitMQ/Celery and shared configuration between servers.
Can you recommend any other comprehensive tutorials that are a bit more high-level than the docs? I'm also interested in replacing our use of Fabric with Ansible - there doesn't seem to be a lot of documentation around running one-off tasks though...
Ansible has worked really well for us at Pathwright. We initially played a lot with Chef and Puppet, as they're two really safe bets. This is kind of petty, but we're a Python shop, so the Rubyisms were a bit jarring. Both were also a bit harder to pick up while short on time. This new thing (at the time) called Ansible caught our eye, and it only took a few days to see that it was a great fit for us.
While Ansible is developed in Python, it doesn't leak through to the UI and the playbooks. It's all just YAML. Learn a little Jinja2, grok the directory structure, and you're set.
The playbook format is simple, and even people without any Ansible experience can pick through and get a basic idea of what's going on. Pythonistas, Rubyists, whatever. It's just YAML.
My only criticism is that the documentation isn't organized very well just yet. This is common with projects that are rapidly growing/improving, and I expect it to get better. In particular, I think the doc index needs to be trimmed a LOT, with more content being delegated to sub-sections: http://www.ansibleworks.com/docs/
Overall, great utility, very helpful community, loads of potential.
I'm not a fan of yaml but I do love Ansible anyway. Another nice thing about it is that it requires nothing on the remote side other than a running sshd and stock python. You don't need to deploy anything like a chef client to each remote system.
True on all accounts. I'm not a YAML fan either, but it ends up being a pretty minor detail for us. It is very convenient not having to run an agent on the managed machines.
The problem with YAML is its reliance on syntactically significant whitespace, a trait that makes the result not seriously human-readable or editable.
Once you can't reliably read or edit the source, you lose the original reason to use plain text -- binary representations tend to be smaller, after you've given up on the idea of hand-editing or interepreting by eye.
This, by the way, is a defect shared by Python source code, one of the few drawbacks to a very productive language. In Python, if a single space gets removed from a source file, the consequences can be severe and difficult to either detect or correct.
The other side of this is that in syntaxes which have some other hierarchical delimiter (such as curly braces), it is quite a common mistake for a human to see one thing, and for the computer to interpret another.
In such a syntax, if a single semicolon or curly brace is missing or misplaced, the consequences can be severe and difficult to either detect or correct.
At least with whitespace, what you see is what the computer also sees.
> The other side of this is that in syntaxes which have some other hierarchical delimiter (such as curly braces), it is quite a common mistake for a human to see one thing, and for the computer to interpret another.
Yes, but in the case of explicit delimiters, the situation can be resolved fully and automatically by counting delimiters -- either there are an equal number of delimiters (i.e. N('{') - N('}') == 0) or there aren't. The computer can be relied on to discover this simple relationship and raise an alarm, one that puts the error before the programmer in unambiguous form.
By contrast, in Python, in many cases the program compiles, runs, and produces a different result than if the error were not present. The program is syntactically correct in both forms, but its meaning has changed. In the Python case, the program may have a different meaning, and produce a different result, than the human intended without an alarm being raised.
> At least with whitespace, what you see is what the computer also sees.
Depending on what you mean by "see", not necessarily so. The programmer may intend something that that program, as written, cannot produce, because of an ambiguity related to the lack of formal delimiters, an ambiguity that causes the computer to understand the program differently than the human, and produce results that the human didn't intend.
All this apart from the nightmare produced by mixing tabs and spaces in the same source file. This may seem an unrelated issue at first glance, but mixing tabs and spaces in a language that uses explicit delimiters is just bad practice, but in Python (and in YAML) it can be a very serious problem. It's why Google's published syntax guidelines forbid use of tabs.
When tabs are used, two standard tabs produce a 16-character-space indentation and two syntactic indentations. When spaces are used, two spaces produce two character-space indentations and one syntactic indentation (by Google's standards, i.e. two-space indentations). Unless the programmer is using an editor that prevents insertion of tabs into the source, this is an accident waiting to happen -- but only in the case that whitespace is syntactically significant.
I think you're cherry-picking your failure case here. Yes - there certainly are errors and use cases for which braces work better. However, there are also errors and use cases with braces for which whitespace works better.
You seem to focusing on one set, and ignoring the other.
"For real"? I write Python constantly. Speaking from a purely statistical and probabilistic standpoint, it's very likely that I've written more Python code than you have. And I don't have whitespace issues in my own code. But by reading the Web I find that it's a problem for many.
Find an example where some student's C or C++ program fails because, as the following examples proclaim, the source file contains a mixture of tabs and spaces.
I'm a bit late to this discussion, but it's worth pointing out that JSON is a legal subset of yaml, so if you want to use a more well defined format, you can. Plus, you get comments for free then.
True in a sense, but false in a more important one. JSON removes the syntactic dependence on whitespace -- one can parse JSON unambiguously while ignoring the whitespace that separates sections, relying only on explicit delimiters:
This makes JSON a superset of YAML, not a subset. JSON has properties that YAML doesn't, properties that remove the ambiguity in YAML's syntax, properties that answer my original objection.
> so if you want to use a more well defined format, you can.
Indeed, I agree. It's true because JSON corrects the primary defect in YAML. YAML was originally designed to be terse and human-readable and -editable, but at the cost of syntactic ambiguity when scanned. JSON solves the problem by reintroducing the delimiters that YAML removed.
Ansible is probably the best configuration management tool out there. But there is always room for improvement. Better docs would be great. I would like to see jinja2 templates replaced with something much, much better. And there is still more work to do for cloud applications.
I definitely don't think that they are bad, just not great. It's not even really a lack of content like with the Salt docs, it's more of an organization issue.
Ansible has way too few examples and documentation going around it. The supplied examples are way too simplistic and do not mirror any real life application at all.
I really hope that the authors create a non trivial project (like hadoop on EC2) which exercises a lot of parts of Ansible, document it throughly and structure it to match real life workflows. It's really, really hard to go around hunting for documentation.
For example, I did not even know after reading the Ansible-EC2 documentation for a few hours that you need to periodically run "python ec2.py" to keep refreshing your inventory in a cache on your disk. I somehow thought (or intuitively expected) all that happened in the background on running the playbook, and just take a few minutes longer - it's not as if any time is saved, since you have to run the inventory script anyway.
Single playbooks that launch an EC2 instance, print hello and shut it down are not the typical use case.
well yes, but it is hard to learn from it. For comparison take a look at this - https://gist.github.com/marktheunissen/2979474 [pedantically_commented_playbook.yml].
NOTE: yes, I did note mpdehaan's comments that this yml is old. However being a beginner in Ansible, I learned a LOT from this.
The OpenShift example is a bit funny - I had asked this on the forums as well. For example the play to shut down a running cluster first attempts to idempotently launch it, causing the appropriate variables to set and then uses it to shut it down. There is another way to do it without doing this. Again, I'm not advocating approach A vs B - I just dont know which is The Right Way and why.
As I said, it is really really hard to learn non-trivial workflows in Ansible. I'm still plowing ahead, but I strongly suspect that is because Ansible has been getting all this support in HN and it is right now the flavor of the month.
Yeah, I don't think everyone shares that opinion, but I do agree we need a better index and a few more examples.
We're always working to improve examples however, and one of the things we're going to be adding is a lot of vertical guides that show some (especially provisioning) workflows from start to finish.
One of the things that is really going to help is the launch of Galaxy in the next few weeks, which is a web site for everybody to share roles, so you will have tons of easy ways to find bonus examples.
Last year, I have deployed ~20 machines with puppet (some virtual, some physical) which also was my first attempt into automatic configuration (better late than never I guess).
I'm reasonably happy with the outcome and it has certainly paid off the learning effort.
But when I learned about Ansible a few months ago and after reading the documentation, I was blown away when I've seen Ansible practically fixing all my gripes I have with puppet.
Audible really feels like the perfect piece of Software for my use case:
For one, I don't have that many machines and the amount of them changes very rarely, so I don't really get much benefit from the central puppet server. It is however one additional piece if software to maintain.
Then, by being able to push a config from everywhere with Ansible, I could get rid of a few scripts that now update the puppet master from git checkouts and then tell the other machines to fetch the config (the clients are not running the agent in daemon mode because I like to have full control over when the deployment of configs happens)
Next, Ansible has a very cool facility to do one-off things across multiple machines. Yes, that's probably a dangerous shoot-yourself-in-the-foot feature, but sometimes, my feet need shooting at and I'm happy if the tool provides the gun instead of myself having to build the gun myself if you pardon the metaphor.
Next, I love the feature that managed clients practically need nothing installed on them to be able to manage them. And: Ansible brings everything it needs to bootstrap a new client. That' say other script I would not have needed to write.
Finally, I like the much bigger library of included modules as that would keep the repository I have to keep smaller and constrained to just the actual configuration of my machines rather than infrastructure to just be able to express the configuration.
Now if only I had time to convert our puppet setup.
Check out my Ansible Playbook that installs a Hadoop cluster (running on Java 7), with Ganglia, Fluentd, ElasticSearch and Kibana 3 for monitoring and centralized log indexing. https://github.com/analytically/hadoop-ansible
In all this talk about Ansible, Puppet, Chef, I wonder why nobody mentions cfengine. When I explored configuration systems my conclusions were Ansible for small systems (<100 nodes) and cfengine3 for larger systems. cfengine has very little dependencies (no Python/Ruby/etc) and a solid theoretic foundation (Promise theory etc).
I feel like for years we've bastardized chef to execute like fabric, using littlechef, custom attribute smashers, and git. Now with ansible this a first class experience, very cool guys!
I see that it's licensed GPL, but then the "pricing" page says the open source version can be used on up to 10 boxes. Is this a code restriction (which could be circumvented) or is it an additional license restriction (which the GPL expressly disallows) ?
Nope, Ansible is GPL without restriction like that.
AnsibleWorks AWX is a commercial UI and REST API solution that sits on top of Ansible -- and 10 nodes are in fact free there, Ansible is all you want! The pricing is for AWX beyond 10 nodes, as well as included support. (You can also get support if you don't care about AWX, but AWX is included).
Cool! thanks for the clarification. I've been trying to wrap my head around Salt for the last 2 days, and this looks really nice. I think I might have to try it
I'm working on a large project running on AWS. We're building an n-tier architecture with many roles in auto-scaling groups.
We wasted the first couple of months writing wrappers around SSH to let us provision servers with salt, which if we'd been using ansible would have been completely unnecessary. We did evaluate ansible at the start of the year, but at that time it didn't look quite as mature so we went with salt.
I've used ansible myself and it's far simpler than salt, and quicker to get started with. Admittedly I haven't used it to deploy the same complex software as we are doing with salt, where orchestration is fiddly. Overall, the thing that lets salt down is its appalling documentation, poor support (the IRC channel has been practically useless) and complex terminology (what's the difference between a module and a state? Who knows...).
Salt does have one thing in its favour though - the ability to run things in parallel. For example a new node can request itself to be configured which is very useful when running in auto-scaling groups. I think this is coming in ansible, but it wasn't at the same level as in salt last I checked.
With salt, when a new node comes up (in an auto-scaling group for example), it can request itself to be configured. Once finished it can fire an event to the saltmaster which can trigger it to reconfigure other nodes. This, for example, means when a new web server comes up, the app can be deployed to it and the node can then instruct the saltmaster to update the load balancer configs to send traffic to it. This event system is powerful because it lets multiple tasks take place in parallel, but it is complicated.
If ansible had an intuitive way of running tasks in parallel (with locks through critical sections, the ability to target a subset of matched nodes (to back up a single server in a pool for example), etc.), then I think it'd beat salt hands down.
For the majority of cases I'd definitely recommend ansible over salt. And as for chef/puppet after using them both on earlier projects, we started looking for something else that could do orchestrations as well as provisioning and that had saner authentication when running in the cloud.
Ansible has lots of parallelism support. From --forks, where you can specify parallelism on the push (Rackspace does this to configure 10k nodes at once in production), to ansible-pull (massive scale out), to the provisioning callback feature in AWX that is just perfect for AWS autoscaling, all of that is definitely there.
If you're talking about multiple parallel playbooks, just launch two playbooks at once, and you can use things like wait_for to hold out for arbitrary conditions.
Chef will take you much longer to learn and though it has some really nice advanced features, it's overkill if you don't need them. Ansible is fairly straight forward to get started and get what you need done. The only problem is it doesn't work with Windows if you host any Windows boxes whereas Puppet/Chef have Windows support. I personally would first go with Ansible then try Puppet if it's not enough.
To add to the mix. I'm doing both. Ansible for the provisioning and coordination/install/running of chef. And then chef for the recipes because there was already an investment there at the time.
I've been wrapping up playbooks in python unittests[1] but i'd like to find something a bit more expressive and less wordy. I could shoogle my wee testing framework a bit more into shape but keen to see how others tackle.
A lot of folks such as UC-Davis will embed tests in their playbooks. This way the playbook will fail if something isn't ok, easy to do with a rolling update, etc.
Just something like the 'uri' module can be a good way to check for a web service, or 'wait_for' can see if a port is up.
I am not convinced by the advantages of the web gui for deployment against our good old fabric/buildbot/deploy, does someone has a feedback with that tool ?
Anyway, I am always very happy to see OSS projects like that making a living of paid services (or the opposite, whatever ;) ).
Are you comparing the web gui to the command line, or Ansible to fabric? It feels like you're conflating two different things there.
I can't speak for the UI, but I use Ansible to deploy everything. It's been a joy and a lifesaver. I just used it to provision a new server (from zero to full Django stack) in 30 seconds. Fabric will accomplish the same things, but Ansible is much more elegant and terse (by a lot). I definitely, definitely recommend it.
For the UI, we use either ansible-pull or http://www.getinstabot.com, which is a chat room assistant I'm working on in my spare time. Being able to say "instabot deploy feature/discounts to staging2" and have the code running there in ten seconds is fantastic, and only possible thanks to Ansible.
Disclaimer: original ansible author here and I run engineering for AnsibleWorks. First off, ansible itself (which this post is about) is text based. I assume you're talking about AWX, which is coming out shortly in 1.4 form, which is our commercial server -- UI and REST API. (Ansible core is decentralized and doesn't even need a server installation).
AWX is designed around team workflow and wants to be the central hub around your automation tasks. i.e. how do you control who can access what, who's doing what, and make things push button in the hands of people who might not be the ones writing the content. You can delegate the ability of lab staff to manage inventory, but not allow them to deploy software, or let them only deploy specific projects, etc. Same for Dev vs QA vs Ops teams, etc.
I definitely don't think it's something everybody needs, but it's available for those that find it useful, and definitely recommend everybody try it out. In the last release, we added some nice source control integration features so you can still develop in your editor and retain the ability to have the web console and central logging. In the upcoming release, we do some neat things with graphically syncing cloud inventory and provide some bonus console information that is useful if you are working with teams who want to know what's been going on where, and it improves the ability to quickly drill down and see what parts of your infrastructure might need attention.
One of the other features is it's ability to hold onto credentials, so you can share source control, machine access, and other credentials with your team without them needing to actually know those credentials.
If you remove someone from LDAP, they can cease to have access.
Anyway, it's super important that we don't hold ansible core back, so what's going into the product is largely around team workflow, and things that automate workflows in Ansible, just like how Ansible automates your system. It's intended to be able to hand Ansible to other people -- and work better in your overall team context, even people who don't want to be writing playbooks. If you're a 1 or 2 man shop, you might not need it, but try it out anyway and let us know what you think!
Also, here's a blog post about the 1.4 Ansible release, which explains the changelog at a bit higher level:
Haven't used the web gui, so can't comment on it. The core open source ansible product is text-based. A few months ago, I migrated a provision and deploy system from a bundle of fabric and shell scripts to ansible. I'm happy with the results.
Some things are easier to do: builtin templating for things like config files is nice and all the modules I've used work as advertised. Some things are harder: reusing groups of similar tasks is kind of awkward but doable. The main advantage is that the ansible imposes a structure that is necessarily self-descriptive and that makes everything very readable. The docs are great and the learning curve is pretty linear so you can defer messing with advanced features until you need them (at that point you'll probably be happy they exist).
I don't want to hijack this thread, but I've found parameterized roles fit my use case perfectly. However, a couple related things caught me out earlier tonight: both the apt_key 'file' parameter, and 'with_file' seem to ignore the roles search paths. I have an apt role which handles installing the package keys, adding and updating repositories, and installing a list of packages. Passing e.g. 'nginx.key' from the nginx role wouldn't work, as they looked only in roles/apt/tasks rather than roles/*/files as I expected. Passing a full path worked. (If I remember correctly! I'm a little hazy on the exact details now, although if this isn't as expected I can do some legwork tomorrow and raise a bug.)
yeah shoot the list an email on this. All the lookup plugins should use the roles search path already, but it could be that one got missed. I'd probably want to see a snippet of the playbook to understand a little better too. Thanks!
Hi mpdehaan. I hadn't seen parametered roles - maybe I just overlooked them. I did use a lot of parameterized includes but roles would be more flexible. Thanks for the heads up.
More and more people are moving from projects like chef to ansible because people need a way to manage servers without the need of too much server dependencies, it is so obvious and I really can't understand why chef developers can't see the needs here.
A lot of great new features in this release - very excited to see the tool become more and more powerful.
I only came across Ansible this summer, but it really blew my mind. I've been doing systems contracting for a few years now and Ansible is the most valuable tool I've come across for systems management so far. Now, rather than have to use (and train my clients on) multiple tools, I can have them just learn one solid tool that can handle orchestration, deployment, configuration management, server inventory, etc. I can now do a systems project in days when before the same project would take weeks. Expect to hear a lot more about Ansible in the future.
Ansible isn't awesome because of a feature-list. It's awesome because it saves me time and gives me a lot of power for very little investment. Whether you're setting up a single server or 10,000 nodes, you can use this tool to give you a ton of leverage without having to spend weeks learning complex tools.
That's why I've been focusing on it so much in the last few months.
Shameless (but highly relevant!) plugs:
For weekly community updates and cowsay jokes, sign up for Ansible Weekly: http://devopsu.com/newsletters/ansible-weekly-newsletter.htm...
For a comparison of Ansible with other configuration management tools (implementing an identical project in each tool), see my book on the subject: http://devopsu.com/books/taste-test-puppet-chef-salt-stack-a...
For a brief introduction to Ansible in comparison to Shell Scripts, see: http://devopsu.com/blog/ansible-vs-shell-scripts/