Stephenmccarron.com

I wrote a post way back in March 2008, on the health and evolution of the Irish market for domain registration and shared web hosting.

So, 18 months later, what’s new, changed or have any of my predictions born fulfillment?

The landscape did change quite dramatically in May 2008, when Hosting365 sold the domain registration and shared hosting business in it’s entirety to Namesco, one of the largest hosts in the world and a leading provider in the UK (and part of a group including other providers).

In this regard, my prediction of consolidation bears some fruit, albeit self-fulfilled somewhat! Recent figures have shown that 71% of the Irish market is in the hands of the ‘tier 2′ hosters (the specialist hosting providers). This tier consists of 39 players, with the bulk of the 71% (some 55%+) held by the top four providers. From a pricing and commoditisation point of view, the leaders have been battling on price for the last year, each offering better pricing on plans, more resources, cheaper domains than the other

The transition has led to the new Register365 still leading the Irish market with almost 20%, still quite some distance ahead of the competition, and growing strongly with, I am sure, exciting things planned and up their sleeves. Register365 now offer plans with 10GB space from €1 per month (for the first month, then €4.95 per month), running on a new highly available ‘cloud’ infrastructure. If that’s too much, you can avail of plans with 5GB of space from just €1 in month 1, then €2.95 monthly. With 7 days phone support, Register365 is still the choice for most of Ireland’s Small to Medium business customers and they are growing at nearly 2k domains per month.

Market no. 2, Blacknight with nearly 15%, offer 10GB space for €4.95 as their entry level, and since my last post, have reduced their prices on domains, including .ie domains in particular. They’ve also achieved ICANN accreditation (the second provider to do so, after hosting365 in Ireland) and are pushing heavily in the area of domain registration and are growing at a similar rate to Register365.

Third place is still Digiweb at 12%, who also acquired Novara in the last two years, and is the only Telecommunications / Broadband provider still in the race with regards to hosting. Most of the other such providers are ‘actively’ losing domains month on month. Digiweb offer plans from €9.99 monthly (less if you are willing to pay in advance for a year) and provides services (including broadband and telephony) including website design, managed hosting, IT services ad more. Digiweb also remain the only other host on the list with their own Datacentre and they are growing at 500+ domains per month.

Letshost, not mentioned in my last post, have been competing highly aggressively on domain price (and price in general) with the cheapest .ie domains available online, and hosting with 12GB space for €7.90 per month (less if you are willing to pay in advance for a year) and have almost 10% of the market to their credit and are in position 4, growing by several hundred domains every month.

- Market position data is based on the excellent work of John McCormack at http://www.hosterstats.com/ taking into account that the Register365 figure includes an accumulation of legacy and new DNS servers, as does Blacknight, Digiweb/Novara and Letshost. Despite some ‘claims’ otherwise lately in the blogosphere, the market ranking has not changed as can easily be independently verified using John’s excellent, freely available data.

- Cathal Garvey has done some excellent work tracking the domain-only price for .ie domains here: http://cgarvey.ie/blog/ie-price-comparison/

So, in short, the relentless commoditisation and margin crushing of the domain and shared hosting space continues apace, the consolidation of the market has taken a few major, and a few stuttering steps, but will develop pace over the next 18-24 months as many of the smaller providers see margins leaving the space and growth rates slowing to small single digits per year (if at all). Much of the market outside the ‘big 4′ is in the hands of website designers, developers, etc, who traditionally bundled hosted (many of whom are resellers of the big 4 also) and might be happy to ‘release’ themselves from the stress and inconvenience of managing a small-scale hosting service.

External to the hosting market, the SAAS / Cloud drive is equally relentless. Services like mobile me from apple, pix.ie, gmail and inumerable others, allow you to simply and easily avail of the ’service’ of hosting, without any of the technical know-how and, in most cases, without any cost. For those who want to put up a simple website, host some files, but photos online, have an email address (even at their own domain), services are available and improving all the time that will fulfill those needs without much time or money required. To my mind, this is the real threat to the shared hosting industry – it’s complete replacement by the ’software as a service’ or ‘cloud’ industry. ‘Vanilla’ hosting (where you need to know what you are doing and understand ‘technical’ stuff) will be the realm of designers, developers, hobbyists and enthusiasts (read geeks ) with consumers and small businesses simply availing of the ‘end result’ through SAAS.

I think my conclusions/predictions from 18 months ago still stand, and look forward to keeping an eye on the business as it evolves and changes…

In Summary

* Big hosts will get bigger (as evidenced by the current market share breakdown and growth rates)

* Smaller and undifferentiated hosts will be swallowed by the Big hosts (in progress)

* The market will continue offering more and more (space as well as features and functionality) for about the same price (evident also)

* The specialised SAAS providers (Salesforce, Sugar, Zimbra, Basecamp, etc, etc, etc) will continue growing and start also consolidating. (Clearly evident, cloud is everywhere one looks these days!)

* Few providers will own their own data centres. (not as relevant these days, but still a factor in determining a providers resources and scale)

From the Deloitte Commentary on the Budget yesterday:

Carbon Levy
The Minister confirmed that the Commission for Taxation has been asked to examine how the introduction of a carbon levy might best be structured and implemented to ensure that Ireland’s economic prospects are protected and the vulnerable in society do not lose out.  A firm measure is expected in next year’s budget.

The fact that the Minister is examining the matter in detail and awaiting a more rounded picture of the impact of the introduction of such a levy is commendable and it would be hoped that he would take on board the views of the Commission and other industry bodies before making a final decision next year.

Energy Efficient Equipment
Finance Act 2008 introduced 100% accelerated capital allowances for companies who purchased specific energy efficient equipment.  These included Building Energy Management Systems, Lighting and Lighting controls, and Motors and Variable Speed Drives. Budget 2009 extended the accelerated capital allowance regime to include four new categories.  These new categories are:

• Data server related systems and large energy saving office equipment associated with Information and Communications Technology
• Efficient heating/electricity provision equipment and control systems
• Efficient electrical and control equipment associated with process and heating ventilation and air-conditioning systems
• Alternative fuel vehicles

Any relief for business which commits to the Green Agenda is to be welcomed, particularly in current times when fuel and utility costs are at historically high levels and driving hidden inflation and additional cash outlay for business.

Source: http://www.deloittebudget.ie/

Of particular interest to me is that ‘Data Server related systems’ are now covered under this new measure, allowing for 100% capital allowance right off. For us, it means we could potentially right off our blade/san/virtualisation investments over 1 year, as opposed to the typical 3. It will be interesting to see how this, and the ’spectre’ of the expected Carbon levy next year influence IT spend in the coming months.

Our Cloud Platform already has impeccable green credentials, consuming a tiny fraction of the power of traditional servers and needing even less cooling.

I predict a gradual but accelerating move to virtualisation and cloud technologies, both onsite and outsourced, with more and more companys outsourcing entire IT functions as the costs of refit, refresh and staffing are compounded by carbon levys and penalties on inefficiencies.

Over the last number of years, the data centre space industry has changed dramatically. Way back in 2001, when I first moved into this industry, data centres were are their lowest ebb. Facilities were shutting down, getting mothballed and generally the market was in poor shape. In Dublin, Ireland alone, more than half a dozen facilities were shut down, sold or taken over in the space of 12 months.

Now, things have changed, changed utterley. The hype and hysteria of the dot.com ‘boom’ has been replaced with the heady realism of the ‘late noughties’ and with it, the realisation of two fundamental truisms:

1 – more and more and more IT services are moving into the ‘cloud’ (I mean the internet cloud here, not the new cloud platforms)

2 – the cloud needs data centre space

At a global level, there is an excellent short summary on the data centre dynamics blog, some of the key points of which are:

“The last six months have seen continued expansion of datacenter space in Europe, with much of the growth coming from the established markets in the UK, France, the Netherlands and Germany. The degree of the increase can be seen from the effects of the Datacenter Price Tracker, where average carrier neutral space per rack has increased from 776 euro per month (July 2008) to 865 euro per month (January 2008), an average price increase of 11.5 per cent over the last six month period according to price tracker Tariff Consultancy.”

11.5% increase in 6 months is rather more than inflation, and over the last year or so, the increase has been even more dramatic.

So, are the data centres becoming money grabbing? Quite the opposite! Consumers have been used to articifically low pricing from the hosting world for a number of years now, few remember the prices charged for hosting 7 or 8 years ago (a lot more than today) which were possible because pretty much everything was cheaper. Staff, space, bandwidth, servers, infrastructure were all available at great rates after the ‘bomb’. That artificial situation, which was basically paid for by all the poor souls who funded ‘web 1.0′ has now run out, and providers are having to actually build and invest in their own facilities (as well as paying significantly more for utilities and staffing).

The current costs for data centre fit-out (assuming you already have the physical building) runs to around €1,000+ per square foot of data centre floor space, but, if anything, more and more space is currently being built.

While space at the moment in major markets (particularly London, Amsterdam, etc) is getting very tight, there is also a hugh amount of building underway. Locally, there are facilities either in build, plan or open in Cork, Shannon, Limerick as well as company’s like Data Electronics, Interxion and more planning expansion or new sites and even the incumbent, Eircom, opening a new 100,000 square foot facility in North Dublin. (This is outside the work underway from large corporates, such as Microsoft, who are building a 500,000 square foot facility in West Dublin).

More locally still, here in Hosting365 we’re building out another 8,000 square feet of facility space at our Park West Facility (altho as we don’t provide colocation in the general sense, our model is rather different).

The race is on, fueled by a lack of data centre space, higher returns on that space for developers and operators and the relentless drive of all services into the cloud !

I read a piece this morning from Network World (http://url.ie/cuf) which detailed the challenges and concerns facing adoption of co-called ‘cloud’ or ‘grid’ platforms.

The piece included a quote from Kirill Sheynkman, head of start-up Elastra – “Equipment inside the corporate data center isn’t going away anytime soon,” Companies remain reluctant, for a variety of reasons, to trust the cloud for their mission-critical applications.

I was surprised to read such a perspective, as running on our Cloud right now are ‘enterprise’ or ‘blue chip’ companies like Carphonewarehouse, Tesco, Citijet and many many more, but as I continued through the article I realised the 8 points being raised as ‘blockers’ to adoption have already been addressed by hosting365.

Specifically:

1. Data privacy. Many countries have specific laws that say data on citizens of that country must be kept inside that country. That’s a problem in the cloud computing model, where the data could reside anywhere and the customer might not have any idea where, in a geographical sense, the data is.

The Hosting365 cloud service is maintained in Dublin, Ireland. Currently in one physical facility, it is currently ‘growing’ to encompass two facilities for redundancy, but both are in Ireland, in the European Union, and regulated by Data Protection and Privacy legislation in this jurisdiction.
2. Security. Companies are understandably concerned about the security implications of corporate data being housed in the cloud.

With our platform, the concern is no greater than with traditional colocation or dedicated servers. Our cloud is protected by best-in-class Cisco Firewalls and VPN devices, with comprehensive filtering, monitoring and netflow analysis, 24/7.

3. Licensing. The typical corporate software licensing model doesn’t always translate well into the world of cloud computing, where one application might be running on untold numbers of servers.

Simply track how many servers you are running in real time – we need to do this ourselves so we can report license usage to both VMware and Microsot (as just two examples)

4. Applications. In order for cloud computing to work, applications need to be written so that they can be broken up and the work divided among multiple servers. Not all applications are written that way, and companies are loathe to rewrite their existing applications.

Our cloud platform allows horizontal and vertical scaling that does not force application developers to ‘build for the cloud’.
5. Interoperability. For example, Amazon has its EC2 Web service, Google has its cloud computing service for messaging and collaboration, but the two don’t interoperate.

VMware (our cloud platform of choice) is pretty ubiquitous. You can move VM’s to your own kit, you can convert back to physical servers, you can move to another data centre, you can move between virtualisation platforms. You can even use something cool like VMware Fusion on your Mac laptop to build a Debian server just the way you like, then put that online in our cloud, without having to change anything! By adopting enterprise standard virtualisation, and probably the most mature virtualisation platform available, we’re ensuring zero ‘lock in’ for our cloud users.

6. Compliance. What happens when the auditors want to certify that the company is complying with various regulations, and the application in question is running in the cloud? It’s a problem that has yet to be addressed.

For all of the reasons above, we’ve built an environment that can achieve PCI compliance (among others).
7. SLAs. It’s one thing to entrust a third party to run your applications, but what happens when performance lags. The vendors offering these services need to offer service-level agreements.

We offer better SLA’s on our cloud than we do with any other offering. Want 24/7/365 support, 100% network and power uptime SLA’s and a guarantee of no more than 15 minutes downtime even in the event of physical hardware failures – step right in! Our approach from day-1 with our cloud platform was to build an enterprise solution that could replace the ‘old’ way of building bespoke kit, but still deliver the same or better SLA’s.

8. Network monitoring. Another question that remains unanswered is how does a company instrument its network and its applications in a cloud scenario. What types of network/application monitoring tools are required.

We provide comprehensive network and application level monitoring on our grid, with full web access for customers to the same.

There are lots of misconceptions about what a cloud actualy is (or can be) in the marketplace, but, from the customer interest and take-up we are already seeing, I feel we’ve provided a solid, enterprise grade platform, that can deliver better performance and reliability than dedicated hardware, with the same or better SLA’s, but also offer the flexibility and control of rapid scalability, complete mobility of resources and workloads, all backed up with our dedicated support.

Recent news here in Ireland announced that IBM was launching their first European Cloud Platform in their first ‘Cloud Computing Centre‘ in Dublin. Subsequent coverage revealed a 100 CPU compute grid which IBM are using to test and develop their cloud offering.

In the last few days, Google have ‘entered the cloud scape’ with their new Application Engine, joining the likes of Salesforce.com who offer application hosting clouds, and competing (a little indirectly) with Amazon and their EC2/S3 Cloud services.

What does it all mean?

In simple terms, ‘clouds‘ are the new ‘grids‘ But fundamentally it’s all built on virtualisation.

- Cloud Grid Platfoms in a few simple steps:

1. Build a big pile of servers, with lots of CPU’s and RAM
2. Hang a big heap of storage off the back and hook everything up
3. Put a network connection in front of the whole thing (maybe some load balancing or firewalls)
4. Run a virtualisation platform (like VMware) on top
5. (Optional) Replicate all of the above in multiple physical locations

You now have a ‘virtual’ datacentre full of servers and storage that, with the virtualisation, can be chopped into pieces appropriate for the users requirement.

So, you need 4x 2.33Ghz Xeon processors, 8GB of ram and 100GB of fast storage – no problem – click- there you go.

You need 16x 2.33Ghz Xeon processors, 32GM of RAM and only 10GB of storage for a super database – click – no problem.

Virtualisation means you can treat the cloud/grid/blob of resources like an empty framework, and run your services on top, with full flexibility in terms of all the key metrics (cpu power, ram and storage). You can move all the metrics around in near real time, without having to stop the ’server’ running (try adding a CPU to a traditional running server with the power still on and the server running? Ouch!)

Here at Hosting365, I’m happy to report our Cloud is even bigger than IBM’s!

Here’s the spec:

• 128 HP BL460C Blade Servers
• 256 Physical Intel Xeon Processors
• 1024 CPU Cores (2.33Ghz, 1666Mhz FSB, 4MB cache per core) (Or roughly 2,385.92Ghz!!)
• 4TB RAM Capacity
• EVA 8100 Redundant SAN Cluster
• 20TB Fitted Capacity (in 146GB 15k FC drives)

You can see a photo here (apologies for the otherwise lovely shot being ruined by Ed Byrne, our General Manager)

Our cloud lives within a network architecture which includes HP Virtual Connect Modules (switches) for SAN and Ethernet, meaning all WWID’s, MAC addresses, etc, are all virtualised and portable within the cloud (providing immunity from hardware failure of individual nodes). These switches are uplinked to two Cisco 6509 switches, providing full redundancy, and are in turn uplinked to our Network Core. There are also Cisco boxes performing Firewalling and VPN Termination and F5 boxes for Layer-7 Load Balancing.

The real magic though, is in the VMware layer. Running VMware as our virtualisation platform of choice was the logical option for such a high performance, highly resilient grid / cloud platform.

Vmware’s maturity and robustness, allows us to provide real-time node High Availability (no impact to users if there are hardware failures or issues), dynamically move resources around the grid, again in real time and without downtime, to ensure there are always maximum resources available for all users. In addition, the snap shotting and management features of VMware Infrastructure tools mean we can deliver a truley enterprise experience.

In short, we spent a lot of money on a platform, so our customers don’t have to compromise.

For the same price as a basic dedicated server, you get access to everything above in terms of reliability, scalability, flexibility and security. To achieve an ‘apples with apples’ comparison, you would need to buy two, highly spec’d dedicated servers, a layer 7 load balancer (HA Pair) to ensure a server failure wont take you down (or a load balancer failure) a HA Pair of Firewalls to ensure security, and a small SAN. Traditionally, we’ve built many such platforms at costs into the tens of thousands of euros per month – now you can achieve the same business objectives from €99.95 per month!

Welcome to the world of tomorrow! (for hosting anyway )

This recent article by Pingdom, got me thinking about the changes in the Irish hosting business, specifically in the shared hosting business here over the last 10 years.

10 years ago, the hosting landscape in Ireland was dramatically different. The only providers of hosting back then were Eircom (the incumbent telco) and small independents like Webworld and Digiweb. (My own first Irish hosting account was with Webworld )

In 1997, the Wayback Machine shows Digiweb were selling hosting for roughly €40 per month, which bought you 25Mb of disk space and 1 pop3 email box and 200Mb of bandwidth per day (roughly 5GB per month) [ http://url.ie/9rm ] It’s worth noting that at this time, Digiweb was a hosting company, not a broadband telco as they are now.

At around the same time, Webworld was offering plans from just under €20 per month, offering 30Mb of disk space, 1 pop3 mailbox and ‘unlimited’ traffic (albeit with a once off set-up charge of nearly €65[ http://url.ie/9rn ]

Eircom were the largest host in the land, with pretty much all of the market that wasn’t already hosting offshore, although finding out exactly what they were offering or for how much is rather difficult!

Fast forward 4 years to 2002 and Hosting365 has been launched! [ http://url.ie/9ro ] The basic offering was 100Mb disk space, 100 pop3 email accounts and 2000Mb transfer per month for €12.95 per month. At the same time, Irish Domains were offering 25Mb space, 5 mailboxes and 1GB data transfer for €12.50 [ http://url.ie/9rp ] , Webworld were offering 50Mb and 5 mailboxes for just € 7.50 [ http://url.ie/9rq ] and Digiweb offered 100Mb space and 3 mailboxes for €14.95 per month.

So, skip forward to 2008 – ten years since internet hosting came to Ireland, and how do things look?

Well, for a start, the providers that were here ten years ago are still here, albeit Digiweb now a Telecommunications company (that does some hosting on the side) and Webworld and Irish Domains relatively small providers ( in about 11th and 8th place in the market, according to WebHosting.info ) The current market no. 2 – Blacknight, began in 2002 as a web design and hosting service provider (they dropped the web design soon after), and was for a number of years a customer of Hosting365, striking out on their own around 3 years ago.

In terms of the competitive landscape, Hosting365 rose rapidly to dominance, now with over 35% of the market, and larger than the next 5 or 6 providers combined and continuing to grow more than 4-5 times faster than the next fastest growing competitor. (Following a market evolutionary style common with many EU countries).

So, what do hosting plans look like now? Well, we started offering 5GB space, 50GB transfer and 250 mailboxes for €3.95 per month about a year and a half ago – this is for a fully featured plan with lots of domains, databases, etc. If you take a look around the sites of Digiweb, Webworld, Blacknight you will see a remarkable similarity in specification and pricing.

So what happened in ten years of ‘high tech’ innovation and development? Well, for shared hosting, nothing, nothing at all. It has become the archetypal commodity service. Prices have decreased tenfold in ten years (from nearly 40 euro per month to around 3 euro) while the volumes of space, mailboxes, etc, has increased two hundred times! (from 25MB to 5000MB). It’s worth noting that during those same ten years, one could easily assume a decent rate of inflation, which actually makes the decrease even sharper.

What does this mean? Well, for the consumer, it means you can avail of huge amounts of resources and services for a really tiny cost (less than you’d spend having lunch for two at a coffee shop for a years service). It also means that, to remain competitive and differentiated in a commodity world, customers are now getting 24/7 support, a wide range of supported platforms, tools, free scripts, auto installers and much much more.

For the provider, this relentless shift to commodity means a tightening of margins (in some places a removal of margins, hence you see hosting providers branching into broadband, for example). The cost of buildings, electricity, people and almost all of the things that really contribute to the cost base of a host have increased steadily in the same period and increased dramatically in the case of power. Inflation has been running high but none of these costs have been passed on to hosting customers.

The result is classic commoditisation of a marketplace. Let me give you my ‘5 year prediction’ on the shared hosting business in general.

- The market will consolidate to a relatively small number of companies. A bit like car manufacturing today, there might be a few brands, but behind them will be a small number of companies. The overselling levels and infrastructure requirements will squeeze the market down to a few core providers, with any other providers reselling.

- Also like the car business, there will be a number of successful niche providers, catering to a specific group or demographic (Premium style hosts).

- Services will widen into more ‘application’ style delivery. As all services move to the cloud, the provision of a small businesses website will become more intertwined with the delivery of email, collaboration, online business, crm, intranet, extranet and other tools that will be delivered as a service. All built on the same building blocks of ‘webspace, email accounts and bandwidth’ but sold for specific purposes.

In Summary

* Big hosts will get bigger

* Smaller and undifferentiated hosts will be swallowed by the Big hosts

* The market will continue offering more and more (space as well as features and functionality) for about the same price

* The specialised SAAS providers (Salesforce, Sugar, Zimbra, Basecamp, etc, etc, etc) will continue growing and start also consolidating.

* Few providers will own their own data centres.

We’re a hosting provider, a ‘citizen of the internet’ and, registered with RIPE, we maintain a moderately large pool of IPv4 IP addresses (many tens of thousands). You may or may not have heard stories about IPv6 (the new, sooper dooper IP number system) that is planned to replaced IPv4, particularly as IP 4 is due to ‘run out’ of address space relatively soon.

The reality though, is that there is sufficient IP space in v4 for current requirements, and, even if the ‘free pool’ runs out (some analysts estimate as early as 2010), there will still be lots of IP’s available and being traded between providers. What’s more, due to incompatibilities between the two systems, there will be a fairly substantial IPv4 only internet for at least the next 10-15 years.

Fundamentally, IPv6 doesn’t really do anything differently to IPv4. It doesn’t eliminate NATs, it doesn’t reduce routing load, traffic engineering is the same and while large, the IPv6 space is not infinite. To make things more difficult, there is not way to incrementally deploy IPv6; -everything- must be changed all together, all the way from the back-end servers to the front-end routers for it to work. That also means the ISP’s and hosting companies need to change their provisioning, billing, monitoring, measurement systems and all of this is only possible with support from all the vendors involved.

Early adopters and Pioneers are moving cautiously in this space,  actual traffic is extremely small and several things need to be ‘fixed’ before we can start the ball rolling in earnest (as a small example, Windows XP doesn’t support DNS queries over v6, therefore won’t work in an IPv6 only environment). Another simple example is SMTP – the internet’s killer app – Email. Until the whole internet is IPv6, all SMTP servers will need dual-stack relaying (otherwise IPv4 mail servers couldn’t sent to IPv6 server and vice versa).

Don’t get me wrong, we need to move to IPv6 and we need to start that process as soon as we can, but we need to keep an holistic, end to end view and filter the marketing hype or we will end up with a fragmented, kludged internet (and then what will we do with IPv10 )

In the last few months, we’ve all seen various providers and services have problems.

http://url.ie/8xh documents a long list of infrastructure issues with some ‘big name’ hosts, some of whom market their ‘zero downtime’ networks – you can see our own status site for an issue we experienced this week – http://www.hosting365status.com – and more recently Amazon S3/EC2 suffered an outage – http://url.ie/97n

Personally, I hate seeing providers in trouble. I’ve been through the same, I know how it feels and it’s not pleasant. You stress, worry, fret and get not a little angry after having planned, invested, planned more, invested (a lot) more and then planned some more and yet still things go wrong.

Professionally though, I am delighted when issues like these happen. Not because I am some kind of masochist, nor do I get pleasure from another providers problems, but because it helps educate the customer.

Customers want 100% uptime of hardware, networks, datacentre, chillers, staff, they want 5 minute responses to their questions 24/7 and they want it all for less than most of them pay for advertising the service they are running online.

This isn’t a rant, I’m not saying ‘customers’ are dumb’, I guess I’m just reiterating the one truism in this business;

- stuff breaks

No matter what you spend, how much you plan or how good your staff (inhouse or outsourced) are, there is no getting away from that fact.

You can invest in multi-site, cloud based, real-time-replicated systems, and reduce your risk of exposure to tiny percentages, but, as Amazon just found out, a simple software glitch can happily replicate itself and upset those plans.

Service providers can certainly do their part. Hosting365.com is about to launch our new ‘Cloud’ Platform, which offers a huge grid of computing and storage, built on world-class HP Blades and SAN, running a super resilient Cisco network and leveraging the best in class virtualisation technologies HP and VMware have to offer. With offsite replicated DNS and load balancing, and multiple ‘clouds’ in multiple datacentres on separate networks, we are getting close to that 100% target – and are doing it at a price point that doesn’t break the bank. Customers need to also review their own continuity plans – do you have backups for the backups of the backups?

Customer support.

As a service provider, it’s the one thing that causes the most stress, hassle, grief and pain (both for me, my team and my customers). The old adage ‘Wouldn’t life be great without customers’ rests probably on the lips of many service providers

Customers who need to be trained or educated; who are not familiar with the systems or processes; who are not happy with perceived or real failings or mistakes; who bemoan the ‘crap’ service on forums, blogs, twitter, etc, etc, etc.

Every service provider has seen it, every provider gets the complaints and suffers the indignant customer on the phone or has to deal with the irate customer on public fora, be it right or wrong.

Personally, I’m ‘happiest’ (and I do use the term loosely ) when I’m getting to hear customer issues.

To use another adage – ‘if you think everything is going well, you clearly haven’t a clue what is going on!’

No one is perfect, certainly not in service industries where even huge companies, devoted to service, can get it wrong and upset the customer or deliver a poor experience. It’s surprisingly easy to screw up, for a system to break or act in an unpredictable way, or, simply, for a customer care agent to ‘have a bad day’ and the customer to experience that.

Not for 1 second do I make excuses for these things. The only way to learn, to grow, to improve, to truly become a great service provider, is to take time and listen. To digest and understand the problem, the failing, the screw up, and work to improve it.

Some concrete examples from hosting365… We suffered a (fairly) high profile power outage in our main facility about two years ago .We analysed, studied, brought in the experts, listened to our customers and spent over 600k in the 8 months that followed building a new Substation, installing new transformers and building a brand new MV Switch Room for our facility. We can now stand up tall and proud and guarantee we’ll never suffer the same issues again. In the same vein, we’ve spent hugely on skilled staff, infrastructure, networks, kit, etc, etc, to ensure our customers get an experience that exceeds their expectations. In fact, as a privately held company, we’ve spent over 4 million euro or so on continuously improving our service and growing our team.

The key issue though, is that unless a customer actually comes here, physically to our datacentre, they don’t see much of this work. Yes, things are stable and reliable and everything stays online, but that’s a given these days. From the outside world’s perspective, the same kind of reliability can ‘appear’ to be provided by hosts with a single server rented from a decent data centre in the USA. Unfortunately, and as I found out about two to three years ago, once you exceed a certain scaling point as a service provider (I believe around 12-15 staff) you can no longer ‘infect’ the customer with your personal passion and enthusiasm. You just can’t meet everyone that sign’s up, discuss everyones requirements and walk them all through the experience. For that you need to depend on building an excellent team and giving them excellent tools to enable them to deliver excellent service. (I know, too many ‘excellents’ )

I remember answering support tickets at 3 in the morning, standing in data centres until my feet were sore and dealing with customer issues on my honeymoon in Mauritius 5 years ago.

I take my business very personally (perhaps sometimes too much so) and believe very firmly in listening carefully to every single customer issue. The customer who takes time out of their day to give you feedback on a failure or problem is the most valuable customer you have, and deserves your time, your patience and your action. This post was prompted by a local provider who did quite the opposite recently on a well known public forum, shutting down a customer site when they expressed dissatisfaction publicly, a move which horrified me personally. The day a provider stops listening to customers and acting on what they hear, is the day they are finished in the service business.

An article at ENN got me thinking, and deciding to look at our stats (http://www.enn.ie/article/10123837.html)

Reported by IE Internet, they claim Spam is running at 63.3%, based on 34k mailboxes they claim they are scanning mail for.

A quick look at our own systems show the following numbers:

As you can see, we’re seeing clean mail of just 6.7% – indicating Spam, Virus and general ‘junk’ levels of 93.3%

Why the discrepancy? Well, for one, I think we might scan a bit more email, and larger providers who scan even more will no doubt have even more refined numbers.

Spam Filtering costs a lot of money to deliver well, we run HP DL580 servers with 4 physical CPU’s and oodles of RAM to keep up with the effort required to filter all the email, without slowing it down too much (a fine line).

In the last month, we’ve processed just over 75 million mails (generating the percentages in the Pie above). The crazy thing is that we only scan mail for 1,752 domains (as we charge for the service) which is a very very small percentage of the nearly 100k domains we host. While people don’t like spam, few are happy to pay for it’s prevention as a commercial service.