Infrastructure as a Service : Just Compute and Go!

Posted on June 16, 2008
Filed Under cognifide, scalability | 1 Comment

Context: Some Investment Bank, Some large project, Some time ago…

Me: Wait up Scott (Scott is our SysAdmin. Disclosure: We still are drinking buddies!)
Scott: What now Gibbon! Whatever you’re selling, I ain’t buying…
Me: Stop your whinging you miserable jobsworth. I need 10 Linux servers tomorrow for about 5 hours of performance testing.
Scott: LOL – like that’s really gonna happen chump!
Me: I’ve got budget, approval and all the necessary papers, plus fat cat sponsors. Make it happen.
Scott: Look at me!
Me: <sigh>. Do I have to?
Scott: Do you think our datacentre is a freakin’ tardis?
Me: Noooo….but I do expect it have a few servers out the hundreds you claim to spend you time servicing. Don’t be so greedy. Share a little!
Scott: Seriously. I’ve got 2 hands, 10 mission critical applications I need to provision servers for like yesterday and no more space for new machines.
Me: I don’t want new machines. Just recycle existing ones.
Scott: That’s even worse. First I have to find them (talking days here), then make sure no-ones using them (talking months here), and then and only then install all your crap on them. Next day – Get Real! More like next month…
Me: I could get to the moon and back in that time with decent alien SysAdmin in tow…seriously, this is bad news.

Today

1) I don’t have to ask Scott for servers, I can get them myself. Hmmm, I wonder what Scott is doing these day :)

2) I don’t have to wait weeks for servers, I can get them in minutes.

3) I don’t have to buy nor own servers/licenses, I can pay on demand.

“I don’t need my own datacentre policed by some power crazy SA, with Infrastructure as a Service (IaaS), now I just Compute and Go!

Did you pick up on the Wash and Go tag line! One of my favourites eighties commericials but let’s get back to the script. IaaS is here and is picking up traction to point where I would seriously question any small to medium IT shop not thinking about downsizing their datacentre and/or putting to bed plans to create a new datacentre. Having pretty much instantaneous access to compute resources in the form of servers and storage, delivered as a metered service, means that you are now only limited by your imagination.

IaaS enables your systems to scale out in a cost effective manner. Grab more computing resources when you need them, release them when you don’t. Neverland! For development shops, the ability to periodically fire up a test environment and tear it down in a shot is pure candy. Not just test environments for that matter, but UAT, demos, integration, and so on.

The Players

First off the bat are Amazon Web Services with its S3 and EC2. IMO, Amazon are clearly the leaders in this space, despite the early critics telling Amazon to stick to minding their shop! If you want a server or storage, eight out of ten companies prefer Amazon. I’m not going to say any more here because the best place to talk to about is on the AWS Blog.

Another lesser known player is Flexiscale. Flexiscale is an outgrowth of XCalibre that has been in the hosting business for many years. Cognifide have been using Flexiscale for about 6 months now. The platform is new and continually improving. We are actively pushing them to make the management of their servers better. We work closely with their engineers and have built a console to manage our on-demand flexiscale servers. Cogniscale is freely available for download. It is not a supported product, use at your own risk, but if you any requests or feature improvements, please talk to Adam Najmanowicz who is the lead developer. He wrote a blog post about getting started with Cogniscale that is worth a read.

Then you get hybrids that sit on top of Amazon that provide management services such as RightScale. Also, you can of course roll your own IaaS in three easy steps. (1) Buy a server. (2) Install Xen or VMWare. (3) Pick one from the X new servers. Done!

Next Steps

IaaS is not rocket science. The whole point is to make things simpler. However, with IaaS, your SysAdmin is no longer the cable guy sweating under desk, fighting with wires, or banished to the datacentre/server room. In fact, they can focus on doing the stuff they enjoy. Administering infrastructure. Which leads us onto the topic of “Automated Infrastructure”. When a single physical box spawns say 12 servers, you cannot hope of continue to manage this infrastructure manually. People don’t scale. You need damned good tools to manage and monitor the enterprise.

Earlier, I wondered what Scott, our friendly SysAdmin would be doing these days. If he had any sense, he would be looking into tools such as Puppet, iClassify, Capistrano to name a few to help automate infrastructure. Something that is going to be real important to companies that wish to take full advantage of IaaS.

Ruby is the R in Rails

Posted on June 11, 2008
Filed Under cognifide, development, ruby, scalability | 2 Comments

I would love to have attended RailsConf 2008 but alas you can only do so much in a day. Although this was a Rails conference, it seems that this year there was less talk about Rails and more about Ruby. This is hardly surprising really given that Rails is just a web framework. Yep, Rails is just a web framework. And just like any other web framework, its serves as a gateway from the web and into your enterprise. Now, after a year of doing Rails, you’re basically done. Coasting. Yes, there are tips and tricks you can pick up along the way but for all intense purposes you’ve got the badge. You’re done. You know Rails.

Now the more pressing problems and prickly questions around the enterprise, integration and scalability need to be addressed and this is where Ruby is amore compelling story. In fact, things in the Ruby space are really moving forward at break neck speed. It’s not surprising really, given the relatively few Ruby conferences there are each year, that Ruby gets to play out and steal some of Rail’s thunder at RailConf.

So what is going on in Ruby space that’s so interesting?

Ruby Implementations – Where integration is Key

First up there’s JRuby. This is a 100% pure Java implementation of the Ruby programming language. And right there is JRuby’s sweet spot. The ability to seamlessly integrate with the Java platform through Ruby is a big win for some. So running Rails under JRuby and using it as a web gateway into your Java enterprise, together with the ability to leverage mature/known Java tools and the free performance hikes that accompany it has been a very appealing proposition to some. A prime example being the ThoughtWorks Mingle product. Also, having access to the abstract syntax tree which can be shared between multiple Ruby runtimes hosted within the same JVM process is pretty neat and a highly efficient way of managing memory.

MacRuby! As a Mac User I’m excited by this implementation of Ruby 1.9. With MacRuby you can write Mac OS X applications in Ruby without the performance hit typically attributed to standard Ruby and Ruby Cocoa. MacRuby is native to Mac OS X core technologies. Damned quick with all the elegance of Ruby to boot.

IronRuby, backed my Microsoft and sits on top of its Dynamic Language Runtime (check out John Lam’s talk on the DLR), is a .NET implemention of Ruby. IronRuby enables Ruby programmers seamlessly integrate with the .NET platform, in much the same way that JRuby does on the Java platform. Ruby.NET is open source .NET implementation that does not have the same pull as IronRuby.

In fact, the main draw for Ruby developers that target JRuby, MacRuby, Ruby.NET or IronRuby is integration with the Java, Mac OS X or .NET plaforms respectively.

Ruby implementations – Standards are key

Let’s park integration. Let’s focus upon Ruby developers building Ruby apps with an Ruby-oriented environment. Both Rubinius (a spec led Ruby implementation) and MRI/YARV (currently the official Ruby) are competing VMs for running Ruby applications. Everyone started with Matzs Ruby Interpreter (MRI) but now we have many more alternatives. The Yet Another Ruby VM (YARV) project has officially merged into MRI as of May 2008 to combine the best of both worlds to enhance the official Ruby implementation.

RubySpec

But unlike Smalltalk that was really killed by competing implementations, the Ruby community, through RubySpec are collaborating on standards and competing on implementation. This is good. No, hold up, this is great! This will hopefully provide more choice to the developers to select the appropriate Ruby implementation that best fits their solution requirements. Hopefully, this will hold the community together and keep everyone pulling in the same direction. So far, that appears to be the case…

..but then there was Ruby for Scale – Maglev

Maglev is a Ruby implementation built on top of Gemstone’s VM that is written in Smalltalk. Chad Fowler goes into the why and when you would choose to use this Ruby implementation other the others. But the problem is very few people have access to Maglev. It promises the world and come on guys its time to show me the money! The community has seen is a presentation. A damned fine presentation I’m told but Charles Nutter of JRuby is not convinced and other people are already dismissing outright Maglev’s claim to scale. 100 times performance improvement is a big claim. One to watch I guess.

And Finally!

Take a look at the following code snippet:

#!/usr/local/bin/ruby

require 'scalability'

# your code

Now the Twitter guys must be get this all the time. Twitter is growing in size and its architecture will mature and incorporate different technologies I’m sure that best fits their current problems. Time will time. But to say its Ruby’s fault is a little naive at best. That said, wouldn’t it be just great if only you could include scalability in Ruby applications, Java apps, .NET apps, or whatever the platform.

Now, that would be awesome. But the simple fact is you can’t include it, you have design it in. Boooooo say the crowd. Kill joy chant the business. But unfortunately, like so many things in the software industry, you have to go the long way round (read as experience) to get desired results. Because there are no short cuts (read as foolish hacks, or inexperience) on the road to scalability…only blood, sweat and tears! But there are leg ups, tools and techniques and people to assist, but that’s any story for another day…

Forget Cloud Computing. Think As-A-Service!

Posted on June 7, 2008
Filed Under cloud | 3 Comments

S3, EC2, Google App Engine, Gmail, Salesforce.Com are all in the cloud. But what does in the cloud mean? Typically, when you ask people what is cloud computing, that is where the confusion begins. As an industry we always allow the key market makers to spin the bullshit that invariably plunges us into a year long struggle to get a grip of what really is important and what’s not. After a couple of weeks of reading, I can hand-on-heart say that we are now smack bam in the middle of trying to untangle this term we glibly refer to as cloud computing.

Let’s go back a 100 or so years when electricity was not a utility. During this period, different electrical companies offered different electrical services. There were services for street lights, residential lighting, industrial power, street car lights and so on. Also, these companies had different equipment, voltages and frequencies. This accidental complexity required businesses to employ full-time electrical experts that were responsible for liaising with various electrical companies and their services/equipment to quite literally keep the lights on in the building.

Of course, over time, services, equipment, voltages and frequencies were standardised. Knowledge was shared. Electrical companies offered electricity as a utility and a metered service. Companies no longer needed to employ full-time electrical experts nor have on-site, specialised electrical equipment. Now businesses and residential users, like you and me, continue to rely on electricity as a reliable, scalable, dependable and metered service. In fact, we can even piggy back our computer networks over our electricity supply.

No questions asked. Electricity as a service is a given.

Utility computing is offering the same kind of deal for computing resources. Cloud computing is just a commercial extension of utility computing that offers storage and computation cycles as a metered service. Today, you can create an Amazon account and pay for storage and computation cycles using its S3 and EC2 services respectively. Cloud computing goes even further and enables software to be deployed into the cloud thereby providing a platform for distributed computing for the masses.

But here’s the important part. Cloud computing is NOT important, in much the same way that power generators and the national grid don’t interest the everyday electrical consumer. What does interest us are the services, price and relationships with have with the companies that supply electricity. At the moment there are three types of cloud computing services that are currently getting their fair share of air time:

Over the next year or so, we are going to hear more and more about these services as they enter the mainstream. Yes, the term cloud computing will be used, but those people in the know, they will be looking for the specific services that they wish to make the best use of. Those guys will be the frontrunners.

Phew, I’m getting tired and the sun is shining outside. I think the best thing to do is to drill down into each of the above cloud services in a separate post to keep things small and simple. Until then…see ya!

A note to the reader

Posted on June 2, 2008
Filed Under cognifide | 4 Comments

Over the last couple of months I’ve been asking myself where my interests lie and that this should be reflected in my blog. But before I could do that I had to be honest with myself. So, I stood up one day, butt naked, well semi-clad and chanted:

Okay, for those of you at the back, let me repeat, cough, cough:

And before the wise cracks start coming in. I’m not part of the production build team. But please don’t underestimate the weight that those words carry for me. I have nothing but envy for true builders. People that delivery production code but that is something I cannot do and do the other stuff that I need do. So this brings me back to the question, what am I into?

In essence, I’m into everyday people building damned good software. What is out there for them? What crap do they have to climb over to get to the job done? Why is so hard? How can it be made easier? Understandably, these are big questions and as stuff changes. I will need to change with them but that’s what keeps things interesting.

So, am I interested in Java the language? Nowhere near as much as I used to be 10 years ago. I used to live and breath Java. It was weird as I was kinda like married to Java but from around 2001 started having an on and off affair with C#. What about Java the platform, yeah, there’s more interest there. How about C#? Well, more interesting as a language than Java, but not really enough for me to re-install VisualStudio to play with it. Ah, the rise of multi-core programming and multi-tenant systems. Now you talking. This stuff floats my boat and leads into how far is this (infrastructure/service/platform)aas (as as service) going? Very interesting stuff. Hang on, that mozilla stuff, prism, looking for more integration between the web page and the desktop. Kind of cool the idea of tearing off a web page from your browser onto the desktop. Then this RIA, Flex, and so on. But firefox 3, nope, lower down the stack i’m afraid. Agile. Yes. Static/Hybrid/Dynamic languages. You bet. Novell. Errr, no!

That’s it. Just thought I’d let you. I’m going to become more active on this blog but just a little more focussed, that’s all.

Oh, let me leave you with this. I saw this advert on the tv the other night, and being an old timer, it really made me laugh. Enjoy!

Adequate is not good enough!

Posted on May 21, 2008
Filed Under chit chat, cognifide | 1 Comment

A friend of mine, nicknamed “LeToe” for his rather poor footballing skills, is MD of a global telecommunications company. Big business. In a roundabout way, we came to learn that on one of his staff improvement courses, he was caught trying to motivate an employee by chanting “adequate is not good enough”! Needless to say, whenever we see the human motivator LeToe, adequate is not good enough is always a great source of fun and the first thing we throw at him whenever we meet up. What a plonker!

Clearly adequate is not perfect but as Daniel Tenner convincingly argues, perfection does not exist. Take a look at the picture opposite. As a product company you start at A and wish to build the perfect product that you think is at B. But actually, its a C. Over the years, we have learnt that you need to involve the users both early and often. Release small. Release frequently. Keep releasing! On good days, Flickr was releasing to production every 30 minutes. No question, this certainly helps companies built better products. They inspect and adapt, veering away from the white elephant at B and more towards the holy grail at C. All good so far.

Users change. If fact, I’d argue that today you woke up a different person than you were yesterday. So over time, C is really a moving target. Only a fool would assume that users and their needs stay the same. So if you’re looking for perfection in your products, you’re pretty much going to be chasing your tail. However, if you lower your goal bar slightly and acknowledge this, what you can do is build a better product that strives to meet its users needs but without comprising itself. Oh yes, there comes a time when the product is pulled so far off track that it makes no sense its current form. You just need to know when that time is or as 37signals maintain, don’t get distracted in the first place and let your customers move on if the product no longer meets their needs.

Now, swap product for company, apply the same thinking, and you inevitably arrive at the conclusion that there is no such thing as the perfect company. Even Google are starting to realise this…

Busy Java Developer: Stop, Look, Listen and Live

Posted on May 14, 2008
Filed Under development | 4 Comments

Back in the seventies, the “Stop, Look, Listen and Live” UK campaign was used to teach kids road safety. Funnily enough, the Green Cross Code Man, who became the face of this road awareness campaign, went on to become Darth Vader!

It is now time for Java Developers to Stop, Look, Listen and Live! The climate is changing. Java the language is entering middle age. Java is not the first language that developers are picking up. These days when I talk to young non-Microsoft developers, a lot of them don’t know Java and have no plans to learn it either. Dynamic and functional languages are on the rise. The web IS a viable platform for business. More and more applications are moving out of datacentres and into the clouds. Since Java 1.4, the interest shown by experienced Java developers suggests that are not focussing on Java the language. A lot of Java developers have not developed against Java 5. Even less against Java 6. As a Java developer, do you know what’s coming in Java 7? Do you care? Do you, Mr. Busy Java Developer, really care? If so, here’s a couple of things I reckon you should, if you haven’t already done so, think about this year as a Java developer.

The Three Gees: Groovy, Grails and Gant. Groovy is a dynamic language for the Java Platform. In short, building on top of Java, it injects a degree of flexibility in the way Java-based software systems are created and continue to evolve. As a Java developer, you are seriously missing a trick if you do not have Groovy in your toolbox.

Grails is damned fine web framework. The Grails/Groovy pairing brings the same advantages to the table that Rails/Ruby does. Now I’m a big fan of Ruby, for the busy Java developer I think that JRuby/Rails/Ruby is more of a distraction than the Grails/Groovy option. Groovy’s support for Java frameworks such as Spring and Hibernate are too good to ignore. Also, if you already know Groovy, Grails is a no brainer. In fact, at the Java Web Frameworks Tournament 2008, Grails made it into the semi-finals. Not bad…JRuby on Rails won :-)

Finally, we move onto Gant. Which is like Ant but the language is Groovy not XML. With Gant plugins for Maven 2, your build options are trivial. The only real contender to Gant is Buildr which is a damned fine Ruby based build system for Java applications. Again, the trade-off for simplicity in the Busy Java Devleoper life is, if you keep to the Three Gees, you’re covered. Otherwise, you have another build system to learn and require all your developers to know Ruby.

Scala: This is a newbie on the language scene that has been bubbling away since 2001. However, like Groovy it integrates with Java beautifully and provides a nice mix of object-oriented and functional to bring yet more options for solving a differ class of problems for Java developers. For the Microsoft people out here, Scala implementations also integrate with the .NET platform. In this era of distributed computing, scalability and component based development Scala arms the Java Platform with some great tools for addressing these problems. And again, for the Busy Java Developer whose bread and butter is the Java Platform, you cannot afford not to be aware of how Scala may fit into your world. Word of warning, Scala is not for the lazy or faint hearted. You need to put some time into this language before you start reaping the benefits!

Spring Source Application Platform: EJB 3.0 is alive and kicking. EJB 3.0 is a great platform. EJB 3.0 is not future. EJB 3.0 will never be mainstream. After the non-sense of EJB 1.0. The constant public pounding by Roger Sessions @ ObjectWatch (Poor Ed Roman). The rather feeble attempt of EJB 2.0 to rectify the shortcomings of EJB 1.0. The industry couldn’t and didn’t wait for EJB 3.0. Now if we could wind back the clock say 24 months and re-branded EJB 3.0 as HellFire 1.0, there would be no such thing as Spring Source Application Platform. EJB 2.0 created the market and effectively stepped out the race by giving us such long lead times. The guys at Spring have capitalised on this massive window of opportunity and have produced something pretty cool. However, not all is great. They have rather naughtily upset the OSGi people, in particular Peter Kriens. Hopefully, they’ll change fall into line for the good of the community.

I like your Bundle!: This leads us nicely onto OSGi. If you haven’t heard about this as a Java developer, where have been? Please, please, please, add it to your list of things for this year to pick up. It a must. Have you never wondered how Eclipse manages the lifecycle of its components? How you can upgrade stuff easily and without a restart. No messy classloaders. Define, package, deploy, stop/start and manage your components. Side-by-side deployments of versioned components. Essential for 24/7 Java applications. This is OSGi. The guys at Spring have created a Bundle Repository that makes the use of OSGi simple.

Domain This, Domain That: Not essential but an important thing to know about. Domain Models, Domain Driven Design (DDD) and Domain Specific Languages (DSLs). When you start thinking about the Three Gees, Scala, OSGi and so on, how can you do that without a domain model. The DDD is an approach to domain modeling, where DSLs are way to facilitate human to machine communication that invariably operate within the context of a domain.

Right, I’m exhausted. Everyone gets busy. It’s easy to be left in the trenches and not have time to step back and take a long look at what is going on around you. I’ve suggested a couple of things that I think are useful. You probably have more. If so, please leave a comment…

Cognifide goes Google

Posted on May 9, 2008
Filed Under cognifide | 3 Comments

This week we made Google our standard office platform for all mail, documentation and collaboration. So we switched off zimbra (mail) that we hosted ourselves, we are making better use of calendaring now and are tentatively becoming more familiar with their online spreadsheet and document apps. Nothing special there.

As a user, I have been using Google apps for a long time now but having your company run off Google is a completely different experience. Google is a Platform. Out of the box you are armed with a pretty decent set of apps and tools to get you up and running quickly. However, these are by no means enough. Over the next few months I think we need to dig deeper to see how we can exploit the Google platform to create the Cognifide-specific services that we need to make us run more smoothly as a company.

Contender ready….

« go back — keep looking »