Today’s blog focuses on the problem of latency when hosting applications or services within a cloud provider’s network like AWS.

Have you ever noticed the awkward delays on international phone calls? It’s mostly network latency. This is the time it takes for your voice to travel through the air to the phone, be converted to an electrical signal, transmitted to the other phone, converted to sound and transmitted through the air to the other person’s ear.

[stextbox id=”info”]
As a curious aside, there is a “natural” latency in face-to-face communications. When you talk to someone two metres away it takes about 6 milliseconds for the sound to arrive. A local phone call can actually have less latency than face-to-face communications because the sound travels such a short distance through the air to the handset and then mostly travels as an electrical signal at close to the speed of light.
[/stextbox]

Network latency in computing is primarily caused by the limits of the speed of light. Packets can only travel so fast. Other network effects, such as the number of routers and network device utilisation also contribute.

As distributed applications evolve and essentially become an amalgamation of geographically dispersed services, the limits imposed by network latency will become more apparent.

One can imagine in the not-too-distant future an IT manager yelling at a subordinate to “throw bandwidth” or “new EC2 instances” at a poorly performing application and not understanding the real latency problem.

[stextbox id=”info” caption=”Latency and Bandwidth”]
If we imagine a jetliner making a trip from Singapore to Los Angeles, latency is analogous to the time the flight takes. Once a network path is established, latency is essentially a fixed constraint. Bandwidth can be thought of as the number of passengers on board. If you add “passengers” more “packets” arrive per jet-liner, but the flight still takes the same amount of time.

Latency is typically measured as round-trip latency.  This is the time a packet takes to go from source to destination and back again. Round trip latency excludes the amount of time that a destination system spends processing a packet. Typically you need to compare network latency and application response times to work out whether your network is the problem or your application.
[/stextbox]

One way around latency is to make the network path a shorter distance. Place latency-sensitive parts of a system closer together. Consider also which network providers you use as they will have different network architectures and therefore different latencies to different locations. And also look at caching strategies (eg. CDNs, web optimisation) to essentially pre-load data at a location. Finally look at reducing the network-chattiness of your applications if possible.

Latency-sensitive components of your system need to be considered up-front in the planning phase. Once you’ve built your distributed application and locked in your provider agreements with AWS and other cloud providers, your latency constraints are locked in. And once set-up, latency also needs to be constantly monitored for change and its impact on your environment.

In my previous post I espoused why IT infrastructure sucks. Many of you pointed out that “The Cloud” was the answer. Amazon Web Services is the market leader for IaaS clouds so I’m writing a series of posts documenting the hurdles (I think I’ve found eleven!) to overcome if you want to use AWS and experience the wonders of a truly on-demand infrastructure.

The first is the issue of…

Data sovereignty

passport by One-Fat-Man (via Flickr)

Data sovereignty is basically concerned with where data copies are located, how it is used, and your legal control over it.

Company data sent to a cloud provider can be located in, or transmitted through, multiple countries with different laws. The data has a different legal posture at each stop along the way.

Countries typically have laws governing Data Privacy, Compliance, Retention, Availability, Breach Notification and Data Subpoena powers. But these laws rarely match up with each other.

Privacy generally relates to personal and confidential information and examples include health and financial information. Different countries have different laws for protecting privacy of data records and different requirements of its people and organisations.

Compliance refers to standards and laws such as Sarbanes-Oxley, PCI-DSS and corporations laws. The effect of compliance is to encourage best security practice on an IT operation including physical security, security processes, system configuration and auditing. Compliance is difficult enough to achieve on-premises. Achieving this across a global cloud infrastructure can be brain-numbingly complex and difficult to audit. See this article about PCI-DSS compliance in the cloud.

Data Retention is about being able to recover company records from a “long” time ago.  Typically an application used to create the data records will have been retired or upgraded. The data records will therefore not be retrievable without firstly recovering the version of the application that generated them. This problem has been solved in some respects by data-archiving systems that regularly migrate data in a readable/standard format. But currently archiving is immature on Cloud platforms like AWS. Cloud applications are inherently upgraded in a rapid cycle and end-users won’t have the ability to recover old versions of the platform for data access purposes. The questions to be asked are: Can I get my data out regularly in a standard format? And will I be able to access this information many years down the track?

Data Availability is more complicated in a cloud environment primarily because there are more entities involved in providing a cloud platform. The legal availability requirements of data need to be understood and contracts with providers carefully assessed so that it can be met. For example if a customer’s banking records were hosted in an overseas AWS region and became unavailable, where is the accountability?

Breach notification laws vary around the world. If your data is stored in Amazon and there is a breach, does Amazon have to disclose the fact by law? If your CIO comes and asks you if your cloud data was breached, would you shrug your shoulders and say, “I don’t know”? The US and Europe have had breach notification laws for several years while other countries like Australia have lagged.

Data Subpoena is a most interesting sovereignty issue. Nation states have used economic reconnaissance for advantage for eons. States and their corporations are rightfully concerned that another sovereign state could access their data. A current example is the US concern about Chinese hacking efforts. Another is that Amazon, being a US company, falls under the Patriot Act and must secretly disclose information to the US government if requested, even if the data is not physically located in the US. Other countries have similar laws for national security reasons. There is reasonable suspicion that economic espionage will be used for advantage and therefore represents a sovereign risk.

Data is the currency of business. When your data leaves your premises, you potentially lose control, oversight and therefore business advantage. How do you solve this problem? What controls can you put in place? Please share you thoughts