The new version of Async Http Client 1.1.0 is now available and for Java and scripting language like Jython, Clojure, JRuby, and Scala.

The new version of Async Http Client 1.1.0 is now available and for Java and scripting language like Jython, Clojure, JRuby, and Scala.

Very interesting.  I've wanted to fork Apache HTTP Client and make a NIO-based async adapter forever.  I'm glad somebody has taken this on!!!  I'll incorporate it into RESTEasy and get some JAX-RSish interface going on top of it.

Question: Are you able to avoid spawning threads on the client?  This is what I hoped a NIO-based HTTP client would be able to do.  Thanks!

 

Very interesting.  I've wanted to fork Apache HTTP Client and make a NIO-based async adapter forever.  I'm glad somebody has taken this on!!!  I'll incorporate it into RESTEasy and get some JAX-RSish interface going on top of it.

 

There was no need to fork Apache HttpClient. There has been a prototype of asynchronous HTTP client based on the same code as HttpClient 4.0 for more than a year. While a few people expressed interest in it, no one was willing to commit to investing time and effort into the project.

In the overwhelming majority of cases NIO makes little to no sense for client-side HTTP. The classic I/O performs much better in terms of data throughput, scales well on modern JREs with up to several thousands concurrent connections, is much simpler to manage the memory footprint with and _massively_ less buggy compared to NIO. There is really just a few legitimate where the use of NIO pays off, mainly in HTTP gateways and proxies that must be able to deal with thousands of outgoing high-latency HTTP connections.

Oleg

 

Hi Oleg,

not sure I agree with your statement about performance ;-) So far there is a lot of projects moving to AsyncHttpClient because of the blocking nature of Apache Http Client. Soon I will add an AsyncHttpClient's new provider based on Apache Http Client (so faking async by spawing threads) and then we will be able to do real performance comparison, comparing the same API with different provider.

A+

-- Jeanfrancois

Hi Oleg,

not sure I agree with your statement about performance ;-)

 

You are very welcome to disagree. ;-) Are you sure you are the first person whose mind crossed the idea of benchmarking the performance of classic i/o vs NIO based HTTP transports? HttpCore had a NIO based backend for years. Jetty had a fully asynchronous HTTP client for years. There is a reason HttpClient is still based the classic i/o model. For fringe cases (HTTP gateways / proxies) there is HttpCore NIO.

http://wiki.apache.org/HttpComponents/HttpCoreBenchmark

 

So far there is a lot of projects moving to AsyncHttpClient because of the blocking nature of Apache Http Client.

 

Sure. Some people move onto the next 'cool' thing all the time. Not long ago it was AsyncWeb.

 

Soon I will add an AsyncHttpClient's new provider based on Apache Http Client (so faking async by spawing threads) and then we will be able to do real performance comparison, comparing the same API with different provider.

 

Right. You are going to put an additional HTTP abstraction on top of an existing HTTP transport. I am pretty sure you can further 'optimize' this setup to get the results you want.

Cheers

Oleg

 

 

A+

-- Jeanfrancois

Hi Bill,

on the client side you don't need to spawn any threads. Thread are spawn internally by the provider (Netty) only when the response cames back. Depending on your AsyncHandler implementation (should be non-blocking and avoid doing heavy computation), the number of thread can be pretty small.

Thanks

-- Jeanfrancois

Have you done any comparison against httpcore/httpclient? I'm looking forward for some way to optimize an http services gateway and would be interested in raw performance numbers and benchmarks

Salut,

no performance made so far as we are still adding features, but like I said above, it will be pretty easy to add a new provider based on Apache Http Client and do some comparison. So that will comes in the next couple of months.

A+

--Jeanfrancois

 The fact that the classic Java i/o consistenty outperforms NIO is well known.

http://developers.slashdot.org/article.pl?sid=10/07/27/1925209

There can be _architectural_ reasons to choose NIO over the classic I/O for some highly specific use scenarious, but not performance.

The problem with the old I/O is that it is, well, old. One cannot write flashy blogs about old stuff.

Oleg

 The fact that the classic Java i/o consistenty outperforms NIO is well known.

http://developers.slashdot.org/article.pl?sid=10/07/27/1925209

There can be architectural reasons to choose NIO over the classic I/O for some highly specific use scenarious, but not performance.

The problem with the old I/O is that it is, well, old. One cannot write flashy blogs about old stuff.

Oleg

The presentation you link is comparing server implementations, not client implementations.  A server would have to do context switching to a thread pool in either case, at least in Java (Erlang is another matter).  For a NIO client, the executing thread already exists and there's no need create and context switch.  Granted I did a simple test benching Apache HttpClient 4.0.2 vs. AsyncHttpClient 1.1.0.  Apache test would spawn N threads to do N invocations and join on those threads (using a ThreadSafe CM).  AsyncHttpClient would create N Futures and do a Future.get().getStatusCode() for each request.  The request was a simple HEAD loopback.

Async[1]: 110ms 100.0% success
Async[2]: 16ms 100.0% success
Async[4]: 12ms 100.0% success
Async[8]: 23ms 100.0% success
Async[16]: 58ms 100.0% success
Async[32]: 45ms 100.0% success
Async[64]: 108ms 100.0% success
Async[128]: 151ms 100.0% success
Async[256]: 1107ms 100.0% success
Async[512]: 500ms 100.0% success

Apache[1]: 213ms 100.0% success
Apache[2]: 7ms 100.0% success
Apache[4]: 9ms 100.0% success
Apache[8]: 19ms 100.0% success
Apache[16]: 41ms 100.0% success
Apache[32]: 49ms 100.0% success
Apache[64]: 57ms 100.0% success
Apache[128]: 122ms 100.0% success
Apache[256]: 219ms 100.0% success
Apache[512]: 449ms 100.0% success

I also added a configurable Server-side sleep to simulate server-side processing and network latency.  With a Thread.sleep(10), the results look a lot different:

Async Wait: 10
Async[1]: 5ms 100.0% success
Async[2]: 11ms 100.0% success
Async[4]: 5ms 100.0% success
Async[8]: 7ms 100.0% success
Async[16]: 14ms 100.0% success
Async[32]: 19ms 100.0% success
Async[64]: 36ms 100.0% success
Async[128]: 58ms 100.0% success
Async[256]: 142ms 100.0% success
Async[512]: 404ms 100.0% success

Apache wait: 10
Apache[1]: 12ms 100.0% success
Apache[2]: 12ms 100.0% success
Apache[4]: 26ms 100.0% success
Apache[8]: 84ms 100.0% success
Apache[16]: 183ms 100.0% success
Apache[32]: 378ms 100.0% success
Apache[64]: 764ms 100.0% success
Apache[128]: 1523ms 100.0% success
Apache[256]: 3055ms 100.0% success
Apache[512]: 6091ms 100.0% success

I'd be happy to send the test if you're interested: bburke - redhat - com

Sorry, I didn't increment AsyncHttpClient wait param correct in my code.  It couldn't ever be faster than 10ms for a 10ms wait :)  Async looks better, but I can't see a client every really doing more than a handful on concurrent requests.  Granted, AsyncHttpClient is a bit easier to do async requests!

Async Wait: 10
Async[1]: 56ms 100.0% success
Async[2]: 15ms 100.0% success
Async[4]: 15ms 100.0% success
Async[8]: 21ms 100.0% success
Async[16]: 17ms 100.0% success
Async[32]: 48ms 100.0% success
Async[64]: 60ms 100.0% success
Async[128]: 66ms 100.0% success
Async[256]: 109ms 100.0% success
Async[512]: 1050ms 100.0% success

> The presentation you link is comparing server implementations, not client implementations.

Oh my god, are you saying NIO sucks on the server side? Seriously, context switching is context switching. Threading strategies are not client or server specific. Sorry.

> Async256: 1107ms 100.0% success
> Async512: 500ms 100.0% success

This is just lovely ;-) Two times slower at 256 than at 512?

> Sorry, I didn't increment AsyncHttpClient wait param correct in my code. 

> It couldn't ever be faster than 10ms for a 10ms wait :)

This made me giggle. Man, If you believe, you can make it happen, right? ;-) Are you sure you got all other things right?

> The request was a simple HEAD loopback

Now, try actually POSTing some data and sending some data back (say ~200K)

> I'd be happy to send the test if you're interested

Sure. Please do. You know where to find me.

Cheers

Oleg

<!-- p { margin-bottom: 0.08in; }a:link { } -->

> The presentation you link is comparing server implementations, not client implementations.

 

Oh my god, are you saying NIO sucks on the server side? Seriously, context switching is context switching. Threading strategies are not client or server specific. Sorry.

Completely untrue.  Again, with an async client, there is no need to spawn any threads other than the client thread and thus, you don't have to worry about synchronization issues.  On the server side, a nio-base implementation would still have to deal with context switching and synchronization as the presentation you linked to suggests.  NIO-based applications can scale better (Jetty, HornetQ have shown that), but you don't start to see the benefits until you have a huge amount of requests (which again, the presentation you linked to suggests).

In other words, an async client is an intiator of requests, while a server is a dispatcher of requests.  Huge, huge difference.

 

> Async256: 1107ms 100.0% success
> Async512: 500ms 100.0% success

 

This is just lovely ;-) Two times slower at 256 than at 512?

The AsyncHttpClient code had varying results, which I agree, is just lovely!  Since this was just a quick, code-in-10-minute benchmark I may have configured it wrong.

> Sorry, I didn't increment AsyncHttpClient wait param correct in my code. 

> It couldn't ever be faster than 10ms for a 10ms wait :)

 

This made me giggle. Man, If you believe, you can make it happen, right? ;-) Are you sure you got all other things right?

BTW, Oleg, you need to relax!!!  I didn't write AsyncHttpClient, it just looked interesting.  Resteasy client framework actually uses Apache HTTP Client 4. 

But, if you want to know the truth Apache HC 4 API is poorly documented, un-intuitive interface and configuration (like how are you supposed to know that you have to close the input stream to clean up even if you don't access the input stream?  No Close/cleanup method?  Come on!!), and the website is out of sync with the code, examples published on website that don't even work or compile, along with, AFAICT, no published javadocs on the website, broken links, etc.  Granted Apache HC is the only client mature enough to use, but its deficienties have me always on the lookout for new client libraries, or, better yet, to fork it entirely and whittle it down to what i want to use it for without all its other baggage and endless abstractions...

BTW, if I didn't make it clear before, I will now.  The only reason I'd consider using AsyncHttpClient would be because it is easier to use.  From my little bench, the performance benefits don't seem worth it compared to the maturity of Apache HC.

Bill,
You are welcome to use whatever tool that suits your needs best. But forking a library should usually be the last resort. You popped up on the list once or twice but never provided any constructive criticism of the API or had any interest in doing any work to improve things.

I'll leave all your assumptions about the client side HTTP be. We can discuss this at length at the HttpComponents dev list if you please.

As to the poor state of HttpClient documentation, you can say what you will, but it is certainly better than that of many open-source projects.

http://hc.apache.org/httpcomponents-client/tutorial/html/

HttpComponents is not backed by any commercial entity. It is a purely community driven project. Small and uncool. Yes, things could be better, but I want to have some time for my little daughter too. As to all other stuff you have said it actually makes little sense. For instance, I always thought that having to close InputStream was a basic Java common practice. But so be it. You started off talking about performance and ended up complaining about API.

Cheers

Oleg

Bill,
You are welcome to use whatever tool that suits your needs best. But forking a library should usually be the last resort. You popped up on the list once or twice but never provided any constructive criticism of the API or had any interest in doing any work to improve things.

I'll leave all your assumptions about the client side HTTP be. We can discuss this at length at the HttpComponents dev list if you please.

As to the poor state of HttpClient documentation, you can say what you will, but it is certainly better than that of many open-source projects.

http://hc.apache.org/httpcomponents-client/tutorial/html/

HttpComponents is not backed by any commercial entity. It is a purely community driven project. Small and uncool. Yes, things could be better, but I want to have some time for my little daughter too. As to all other stuff you have said it actually makes little sense. For instance, I always thought that having to close InputStream was a basic Java common practice. But so be it. You started off talking about performance and ended up complaining about API.

Cheers

Oleg

Red Hat actually supports Apache HTTP Client 3.1 and 4.x as various components, libraries, and projects within our Java ecosystem use it (if we ship it, we support it).  I've wanted to fork it for some time for a few reasons:  1)to gut Apache baggage like commons logging, 2) to gut and tailor it to the resteasy client interface so that resteasy client interface isn't an abstraction on a abstraction on a abstraction, faster, smaller, less jars, and easier to support, 3) to narrow the features I want to support for my user base, 4) To make it easier to configure, 5) and the least important, write a threadless async API on top of NIO.  I haven't done it yet because I just don't have the cycles and I'm not convinced yet that it is worth the effort (this is 2+ years now ;)  ).

I ranted about Apache HC because, well, you seemed a bit abrassive towards AsyncHttpClient and I just wanted to let you know there are good reasons why somebody might want to look at AsyncHttpclient over Apache HC beyond just performance.  Also I've seen other I/O intensive libraries/services out-scale and out-perform their peers that are based on async-io (but not necessarily NIO), so AsyncHttpClient looked interesting.

The connection closing thing was one of the annoyances migrating from HC 3.1 to HC4.

Anyways, peace, no worries, nothing personal, apologies, etc. etc. etc.  I'm glad Apache Http Client is active and mature and maintained by great developers like yourself.  Since I believe REST will become more and more pervasive over time, your work is vital.

Cheers,

Bill

Bill,

It is all cool. I usually really make my best to stay away from discussions about competing projects but you really rubbed me in a wrong way with your remarks about HttpClient.

Just my closing remark. Before forking a library one should always make an attempt to talk the differences with the people maintaining it. Sometimes there are reasons for things being one way and not another. There is a reason why HttpCore was decoupled from HttpClient. HttpCore has zero dependencies (no commons logging), is very lean, and can be used to put together HTTP services with a custom API. Does that sound vaguely familiar? And guess what? HttpCore NIO is thread-less per default (both server and client side) and fully asynchronous in case one likes it that way.

Good luck to Async Http Client. Competition is usually good

Cheers

Oleg

What I'm realling wondering is for the polling case, what is thread usage?  Is a thread spawned per request?  Or can a client avoid thread spawning in the polling case?