Eucalyptus Mobile Edition

Got my hands on a draft press release you might find interesting....

SANTA BARBARA, Calif. – August 26, 2013 – Eucalyptus Systems, the leading provider of AWS-compatible private and hybrid cloud computing software, today announced the immediate availability of Eucalyptus Mobile Edition 1.0. This release includes EucaSpension, a collection of utilities and leaf springs for mobile application developers to dramatically reduce time to market regardless of programming language or octane level.  

"Eucalyptus Mobile Edition addresses the needs of fast paced companies facing growing public cloud costs," said Andy Knosp, Vice President of Marketing and Products, "With advanced off-road handling and the highest towing capacity in its class, Mobile Edition lowers up to 75% and increases test capacity by 4000x. We believe this is the only cloud solution that truly empowers the innovators!"

....<insert Barb Darrow quote>...

Pricing starts at $14,000 after a $2,500 manufacturer's rebate and $1,500 early adopter incentive program. For more information, or to schedule a test drive, please visit www.eucalyptus.com or call 866-456-3822 (EUCA).

....<insert company boiler plate> ....

 

--------------------------------------

The press release above was sparked by a picture of the new hardware for the Eucalyptus Community Cloud.

 

Continuous Integration - Hybrid Cloud with Eucalyptus, AWS & Jenkins

Recently I forked the Jenkins EC2 plugin to address bugs with its support of Eucalyptus and to demonstrate use of both public and private clouds for continuous integration. This was neither an academic exercise or a chance to use "hybrid cloud" in a sentence;  Eucalyptus users are already running CI across Eucalyptus and AWS. 

These users benefit from lowered development costs and greater control / consistency of the dev and test environment. It's the kind of stuff lots of development teams would benefit from. Unfortunately their efforts are locked up because their organizations inhibits contributing back. (*cough* legal department *cough*).....So Vic Iglesias and I built a version anyone can use and in this post I'll explain how you, too, can take advantage of it. 

By the way, if you're just here for the show check out this short video (3:30) demonstrating the updated plugin in action.

Jenkins Primer:

Jenkins is possibly the most popular continuous integration solutions available as open source today. It basically uses a master-slave model by which the Jenkins server (master) may dynamically scale up/down worker nodes (slaves). The EC2 plugin allows the Jenkins server to launch and slaves on the AWS cloud, giving a development team seemingly unlimited capacity.

Using the Jenkins EC2 plugin is very straight forward. Essentially it lets you tell Jenkins about the clouds you want to use. Then you describe one or more images (ami's) Jenkins may launch as needed. There's considerably more to the plugin and I encourage you to visit the Jenkins EC2 Plugin page for more info. Or hit up the cool folks at CloudBees; they offer commercial support with Jenkins Enterprise. The subscription nets you enhanced features, increased scalability and support of major plugins.

Preparation

If you already use Jenkins and have access to a Eucalyptus cloud then this section will be a breeze. On the other hand, if you're not using Jenkins or Eucalyptus then you'll need these resources to play along:

• Eucalyptus Get Started - This page describes different options for getting your own Eucalyptus cloud (or account) going.
• Eucalyptus Documentation - Exactly as advertised.
• Meet Jenkins  - This page describes Jenkins and provides installation options.
• Jenkins: The Definitive Guide - An excellent O'Reilly book covering all things Jenkins. Download the pdf, or purchase online.

I assume you already use AWS and have your credentials handy.

Installing the updated EC2 Plugin

This is pretty straight forward stuff. Remember, if you have a version of the EC2 plugin installed then be sure to back it up. The updated plugin should work as well, if not better than version you're running. But no need to take chances :-)

1) Get the updated plugin

     Download the version used in the demo

     Build from source 

2) Manually install it

      Just copy it into your [JENKINS_HOME]/plugins directory

3) Restart Jenkins

Add a Eucalyptus Cloud

If you already use Jenkins and AWS then these next two sections will get you setup for a hybrid cloud. If you aren't already using AWS in your Jenkins environment then just know that the setup is largely the same for Eucalyptus and AWS. Here's how to add a Eucalyptus Cloud to your Jenkins environment:

1) Navigate to Jenkins -> Manage Jenkins -> Configure System

2) In the Cloud section, click "Add a new cloud"

3) Select "Eucalyptus"

4) Fill in the fields for your Eucalyptus cloud / account

    CAUTION: You must include a trailing "/" to the Eucalyptus URLs, or you will receive the dreaded "Login Failure: all modules ignored" message

 

5) Click "Test Connection" to validate your settings

Add an EMI

Jenkins now needs to know which image(s) to launch when using this cloud. You can use just about any image in your Eucalyptus cloud. Some organizations choose to use a base operating system image that gets software injected upon startup using init scripts or configuration automation tools like Ansible, Puppet and Chef. Others like to bake an image with all the necessary software and tools. Both models work just fine for this example, so figure out which image you want to use and follow these steps:

1) In the AMI section of your Eucalyptus cloud definition click "Add"

2) Fill in the appropriate fields - as shown in this example:

3) Click Save to save the Cloud and EMI settings.

In the demo video you may notice the "Usage" setting is "Utilize this slave as much as possible" and the "Idle Termination" was disabled. One of the plusses to running instances on Eucalyptus is that it costs the same whether you run it for a minute, an hour or days at a time ($0.00). Leaving it up means never taking a hit on slave startup time.

Testing Your Hybrid CI

With Amazon and Eucalyptus clouds added to Jenkins you can easily configure jobs to build in either environment, or in a specific cloud if required. One of the easiest ways to ensure jobs run in the correct environment is to apply labels to the images (EMI / AMI) defined in Jenkins. For example let's say you setup both an AWS and Eucalyptus cloud, each with one image. Then you assigned labels indicating what instance size the image will launch as and a label for which cloud. The labels you'd set for the AWS AMI might be "M1.Medium" and "AWS". The labels for your Eucalyptus EMI would be "M1.XLarge" and "EUCA". You can, of course, define many images and many labels.

Now to tie a job to one cloud or the other you'd simply enable the "Restrict where this project can run" setting, then set the appropriate labels. Using the implied labeling scheme above, here are example restrictions you might set:

   Label Expression          Meaning

AWS                               Only run this job on an AWS instance. No instance size restrictions

EUCA                             Only run this job on an Eucalyptus instance. No instance size restrictions 

M1.Medium                    Only run this job on medium instance size. Any cloud will do.

M1.Medium -> EUCA    Only run this on a medium instance, running on your Eucalyptus Cloud.

Now you multiple clouds, a labeling structure and some rules for directing jobs. Fire off a build and watch the magic happen.

What's next?

It's our intention (Vic and me) to contribute back the Eucalyptus specific fixes. We've already reached out to the EC2 plugin maintainer and hope to have a pull request ready to go shortly. Once that's complete I'll update this blog post and you'll be able to use the built-in mechanism to update the plugin.

Hope you found this post helpful. Feel free to join me on the #eucalyptus IRC channel or Eucalyptus mailing list with any questions.

Edit: Added Caution notice regarding trailing "/".

Eucalyptus Milestone 4 - Notes

The Eucalyptus development team host a sprint review every three weeks or so. The event is open to everyone with an Internet connection and it's recorded for those who can't make the live show. I highly recommend you subscribe to the Eucalyptus Users mailing list to get notified when the next event is scheduled - or follow Eucalyptus on Twitter. In the meantime, here are notes I captured during yesterday's review of the Eucalyptus 3.3 pre-release, Milestone 4.

Resource Tagging

Resource tagging is now available and integrated into the UI. One powerful and simple use here is to assign names to objects (e.g. MarchBackup), rather than only using the default resource ID value (e.g. snap-E708D3H).

So if you assign a "name" tag then it will appear here. There was a question about whether the user will be able to see the ID value. This isn't possible just yet so the UI team will look into how this might be addressed. NOTE: this doesn't mean that it'll get implemented for 3.3, just that the team will at least investigate it

Maintenance mode

Maintenance mode refers to the ability to evacuate instances fron one or more Node Controllers (NC) so that the NC(s). To do this the Cloud Administrator uses the migration tool (euca-migrate-instances) to tell Eucalyptus which NC to clear out and take out of the pool. Eucalyptus then moves instances from the target node to others. Here's what it looks like from the back end.

Hard to tell what's going on here, but here's the gist:

1. Cloud admin runs euca-migrate-instances against a target node
2. The backend looks at the instances and tries to find a new home within the cloud
3. Each instance is migrated.
4. Once instances are all migrated the target node is in "maintenance mode".

The top left window shows the euca-migrate-instances call. The bottom left window shows that we had two nodes in the demo. The rest of the windows show the logs for the controller and each node so we can see things happening in realtime.

Migration over VMWare too!

The demo above was shown running on a KVM backed cloud. The second demo showed this working on VMWare. I highly recommend viewing the webinar replay to see this in action.

NetApp Cluster Mode Support

 Swathi did a great job providing an overview of NetApp Cluster Mode's capabilities, deployment options and sample uses. She then explained the settings required to get Eucalyptus to work with this kind of rig. This part was totally new to me and was covered so fast that I found it difficult to keep up. Those with storage experience would have no trouble. Still, there were a few key take-aways I captured:

To use NetApp Cluster Mode, Eucalyptus

• Must have chapserver - Node Controller will use this to have a more secure connection to the storage device.
• Must define NC paths - the addresses the NC's will use to communicate with the backend storage.
• Must have vserver name - …I missed what this was for...

AutoScaling

This part was pretty cool and I got caught up watching the demonstration rather than taking notes. If you're already familiar with AWS Auto Scaling then you'll immediately understand how to use Eucalyptus' Auto Scaling. In fact, the configuration is the same and you can use the AWS command line tools with Euca.

CloudWatch

Users can enter just about any kind of metrics to drive cloud watch rules at - EC2 and EBS metrics. ELB and others are coming in the next sprint. All the metrics and metric definitions adhere to the AWS spec and just like the Auto Scaling, users can use the AWS command line tools to drive this Eucalyptus service.

It is helpful to note that the CW is a separate component, but currently co-located with the CLC. In the future it may be possible to deploy the separately. Pretty sure the team will wait for the user community to help decide if that's going to be necessary.

Screen shot of the metrics used in the demo:

Here are the alarm definitions (notice the first line showing use of the CW CLI).

And finally, the alarm history:

Elastic Load Balancing

Last sprint we saw the basic flow of ELB, which gave us a sense of how it would function. In this spring the implementation was delivered and ready for QA. It is important to note that the ELB service is actually running as a "protected" instance within your Euca cloud. It is also possible to run multiple ELB server instances. The Cloud Admin will have the ability to start, manage, and stop the special instances. These instances will largely be invisible to Cloud Users.

With that background in mind, here are the enhancements delivered in this sprint:

• Configure Euca DNS service to map load balancer DNS name to a set of load balancer VM's public IP
• Configure and describe instance health check
• Build the ELB EMI to be included in euca install packages.

There will be two different images. One for developers one for production use. The developer image will be easy to ssh into and is a bit bigger to accommodate tools.

The End

That's a lot of progress made in three weeks! Be sure to join the mailing list to keep up to date and get notified of the Milestone 5 event.

Private Cloud - Thanks to the Innovators

I am beginning to believe that 2012 will be the year in which private cloud adoption moves from primarily Innovators to Early Adopters. One might argue that we're already there since Eucalyptus customers include 20% of the Fortune 100. Still, there are easily tens of thousands of businesses that could reap the benefits of private and hybrid cloud.

So why aren't more businesses putting private cloud to good use? Well, it's not lack of interest. We've seen over 25,000 Eucalyptus clouds start within a year, so clearly there is potential demand building up. It must be something else. My guess (and it's just a guess) is that "the cloud" hasn't been well understood and that private cloud software has been, until recently, difficult to install / configure / manage.

The folks willing to take a chance on private cloud have been the innovators. They, as Geoffrey Moore would tell you, understand the potential value AND have the technical wherewithal to cobble together disparate components to create the first few working versions. The innovators have demonstrated that private and hybrid cloud solutions aren't just technically feasible, but actually provide real value to developers, IT and the business as a whole.

Over the past few years we've worked along side them and learned (the hard way) how to build the kind of software that can transform businesses. We've put that experience into education and certification programs. We've vastly improved the setup and configuration experience. We've made all our code open sourced. What the Innovators helped pioneer has made it easier for nearly anyone to spin up their own private cloud.

If you've ever wished your IT resources could be as easy to provision and use as Amazon Web Services, then this is the right time to check out Eucalyptus.

Hybrid Cloud: Public to Private

Prior to joining Eucalyptus I heard a lot of hype around Hybrid Cloud - the combination of on premise and public clouds. Most of the time someone would talk about how businesses were going to run applications behind the firewall (private cloud) and "burst" to the public cloud when demand exceeded thresholds. What I soon discovered was that "cloud bursting" was closer to hype. On the other hand, there were real hybrid cloud use cases that were delivering real value to customers.

This is the first in a series of blog posts in which I explore hybrid cloud scenarios. We'll begin with the model that many organizations started with over the past 5 years - using Public then Private clouds.

Why Start with Public Clouds

An application begins its life cycle within the public cloud for any number of reasons. If you look back 4 or more years ago, there wasn't much in the way of private clouds so we can skip the obvious "there is no private cloud" reason. These days there tend to be two common reasons developers adopt public clouds; it's faster and it's cheaper. Let's consider those in turn.

Time to reboot

First blog posts are the hardest to write and the least likely to be read. So the only point of getting this first post published is to make way for something more interesting to write about.

1 down. Many more to go.