Five expert tips on how to build a distributed architecture

Gathered from interviews with various industry experts, here we take a look at some tips on how to build distributed architecture, from using NoSQL and Hadoop, to grappling with RESTful web services and the CAP theorem.

In keeping with the theme of distributed architecture this month, we figured a few of our readers would be frothing at the mouth to learn some great design and implementation tips. At The Server Side, we aim to please, so we've put together a brief compendium of some of the most clever tips and practical insights we gleaned from interviewing subject matter experts.

#1 Don't be late to the NoSQL party

The reasons why you stay with NoSQL will be different from the reasons why you went there.

Dan McCreary, co-author of Making Sense of NoSQL

Do you think that NoSQL is cool but it's just not for you? Think again. Dan McCreary and Ann Kelly, co-authors of ‘Making Sense of NoSQL', say that this database technology isn't just for companies with enormous data sets. Forget about Twitter and LinkedIn as "typical" use cases. These new systems offer benefits even for Big Data that isn't gargantuan. You apparently get lots of unexpected goodies when you give NoSQL a go. There are dramatically simpler interfaces (no Java or object relational mapping required). You may be able to simply drag XML files into a storage system, start running queries immediately and then get to work creating services. Your developers have their time freed up to build enterprise applications that keep your business flexible in rapidly changing times. That's a lot more helpful than just solving the unstructured data problem. According to McCreary, "A lot of the reasons you stay with NoSQL will be very different than the reasons you thought you went there."

#2 We hope you're good at math

Chuck Lam, the author of Hadoop in Action is our go to guy for all our questions about Hadoop.He sees a bright future for this framework, but says that not every user really understands the limitations. Simply put, it won't do all the work for you. "You need a good understanding of the problem you are trying to solve at a deeper level. For example, people think Hadoop will just scale automatically. They think they can write their algorithms in this MapReduce framework and throw more machines at it and it will scale. It may work most of the time, but when it doesn't you're in trouble. You still have fundamental algorithmic complexity issues. With an algorithm like O(N2), throwing more machines at it isn't going to help, even with Hadoop. You have to solve the problem in a different way."

#3 Are your RESTful web APIs safe?

Can't sleep at night wondering whether your REST services are secure? The first thing you want to do is make sure your web API is relying on HTTPS. This ensures secure communication between the client and the server. We also have some good news. For flexibility authorization, custom jobs are a thing of the past. According to Jerome Louvel, creator of the Restlet Framework, "This year, the proper, standard way to authorize access to your web API is to add the OAuth2.0 for authentication. Before, the standard was really not stable since it was going through multiple drafts. Make sure your RESTful web API is compatible with OAuth2.0 in its final version."

#4 Don't be lazy about problem solving - Put your thinking CAP on

Distributed architecture isn't easy, but blaming your problems on a theory with an obscure acronym doesn't make those problems go away. Dr. Daniel Abadi, Associate Professor of Computer Science at Yale University, is known for pulling no punches when discussing this topic, "My big issue with the CAP theorem is that people are using it as an excuse to build distributed systems that do not guarantee consistency. Nowhere does it say that when there is no network partition that the database need not be consistent. It just says that when there is a partition, you give up either consistency or availability. But when everything is going fine, you don't have to give up either." Of course, consistency vs. latency is a whole other matter and there are tradeoffs to be navigated there that have nothing to do with the CAP theorem. In many cases, it does make sense to ensure low latency for a global consumer audience while giving up a little bit of consistency.

#5 A distributed system is no place for control freaks

This is one of our favorite tips about web services. It comes from Arnon Rotem-Gal-Oz, the Architecture Director of the Enterprise Division at Nice Systems. He confesses that engaging in distributed transactions in SOA or RESTful architectures sometimes means letting go of complete control. For example, you can't necessarily ensure that a transaction can be fully rolled back once it's gone its merry way out into the web. "When you are doing something that is flexible in its nature, the other end of your transaction may eventually be someone else, not your company. You cannot afford to hold logs for someone outside your service boundary." He recommends the Saga interaction pattern to compensate for the failures that inevitably occur. 

Let us know your thoughts on how to best develop a modern, distributed architecture.

Dig Deeper on Web databases and data-oriented design

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.