This post is dedicated to discussing the asynchronous processing abilities of the Servlet 3.x family of specs.
I am going to warn you ahead of time that I’m not an expert at this stuff. If I get something wrong, you have my deepest apologies (and if you really want to be nice, please let me know of my error). With that out of the way, I’ll cross my fingers and get started.
Servlet 3.0 VS 3.1
We are going to be discussing two features in this post. The first is asynchronous Servlets, which are supported in Servlet 3.0 and above. The second feature is asynchronous I/O, which is only available in Servlet 3.1, which as of this writing is the latest and greatest version of the specification. The example code for this post will be found in the servlet_3_async branch of our example project on Github (ZIP file is here). This branch is configured to use version 3.1 of the servlet-api library, so that we can demonstrate both features at once.
Async Servlets
Whenever a request comes into your Java application server, it hands the request off to its own thread for processing. In order to handle multiple requests at once, the server will create and manage a whole pool of threads. When a request is finished, its thread is tossed back in the pool, ready to handle yet another job.
In most cases, this setup works just fine. It only starts to break down if your application finds itself handling too many long running requests. If your code has to execute some long running task - or something causes it to block - then your server is essentially working with one less thread for a long duration of time. Now imagine that several of these tasks are occurring all at once. This could put your server in a scenario in which there are no available threads in the pool. In this case, any new requests have to wait just to start processing, which in turn will cause users even greater delay.
Asnyc Servlets help us mitigate this issue. When used correctly, they allow a long running task to be run in its own worker thread, allowing the server thread to be freed up to handle new requests. When the task is finally finished, it re-obtains a server thread in order to send back the response to the user.
Here is a basic example of an asynchronous Servlet. There are a lot of remarks in the comments:
package net.cmw;
import javax.servlet.AsyncContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
/**
* Basic example of an asynchronous Servlet.
*/
@WebServlet(name = "basic-async-servlet",
urlPatterns = {"/basic-async"},
description = "Basic Async Servlet Example",
asyncSupported = true) // notice the asyncSupported property
public class BasicAsyncServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException,
IOException {
resp.addHeader("Content-Type", "text/plain");
// The AsyncContext object is an async-capable ServletContext.
// We need to make this method call to start async processing.
final AsyncContext async = req.startAsync();
// You can adjust how long the server will wait for the async process to run.
async.setTimeout(500);
final PrintWriter writer = resp.getWriter();
// Create a Runnable that will run in its own Thread.
Runnable runner = new Runnable() {
@Override
public void run() {
// We marked the response's output writer as final,
// so we can use it here in the Runnable.
writer.write("Some content\n");
// Simulate a long running process.
try {
Thread.sleep(5000);
}
catch (InterruptedException e) {
e.printStackTrace();
}
writer.write("more content");
// We *have* to call async.complete() to notify the server
// that async processing is finished, and that a server thread
// is needed again to return the response to the client.
async.complete();
}
};
runner.run();
}
}There are a few things to pay attention to - the asyncSupported property in the class-level annotation, the call to startAsync() near the start, and the call to async.complete() at the end. All of these facets are required in order for this feature to work properly.
Asynchronous Servlets are supported by a new kind of Listener, the AsyncListener. Here’s an example which simply shows the methods available to the Listener; I left their contents empty for the sake of simplicity:
package net.cmw;
import javax.servlet.*;
import javax.servlet.annotation.WebListener;
import java.io.IOException;
/**
* An example of an Async Listener
*/
@WebListener
public class BasicAsyncListener implements AsyncListener {
@Override
public void onComplete(AsyncEvent event) throws IOException {
// This method is called when an async event is finished processing
}
@Override
public void onTimeout(AsyncEvent event) throws IOException {
// This method is called when an async event times out
}
@Override
public void onError(AsyncEvent event) throws IOException {
// This method is called if an error occurs during processing
}
@Override
public void onStartAsync(AsyncEvent event) throws IOException {
// This method is called when async processing begins
}
}Async Listeners are useful for when you may need to perform cleanup or other kinds of housekeeping before, during or after processing.
Things to Keep in Mind
While not shown in this post, I kept the old LoggingFilter Filter running in this example. If you take a look at the code for the Filter, you’ll notice that it has one small change, in the form of the asyncSupported property added to the class-level annotation.
Let’s make one thing clear - you cannot write an async Filter, much as that might seem useful. However, if you use asynchronous processing in your Servlet, then you must add the asyncSupported property to any Filter classes which will run before it in the Filter Chain.
Speaking of Filters and async, there’s another interesting wrinkle you should know bout. In my example Servlet, I used the method req.startAsync(). There’s another form of this method, with the contract startAsync(ServletRequest, ServletResponse).
What’s the difference between the two? Why would you pass the request and response into one, but not the other? Isn’t that redundant?
You know the drill by now - once again, the answer is ‘no’. If you use the startAsync(ServletRequest, ServletResponse) version, then async processing will occur using the request and response that is passed into the Servlet. If you remember our post on Filters, you’ll remember that either of these two objects could be a wrapper object passed in from a Filter. If on the other hand you use the startAsync() version, then processing will occur using the original, unwrapped request and response. This is very big difference, so keep it in mind when choosing which one to use.
When to Use This Feature?
I’ve been pretty vague about when you might need to use an async Servlet, using the term “long running process” and nothing more. In reality, that description is too vague. Async Servlets are useful if your Servlet has to make a call to an external REST service, or if it has to conduct some CPU intensive task. In other words, use them in situations where there’s a good chance that a bottleneck could occur.
Async I/O
In writing this post, I had to do a lot of research on Async I/O. At first I didn’t understand how it was different from an async Servlet.
It comes down to the difference in how I/O operations can block compared to CPU bound operations. Imagine the following scenario - someone sends a POST request to your REST API. The body of the request is rather large, and there’s latency on the network. This means that the request body arrives in bits and pieces. Your application is fast, so it consumes each bit of incoming data faster than the next batch can arrive. If you’re using synchronous I/O, your sever-side code will sit and block, doing nothing while it waits for the data to arrive.
If we use an async Servlet, things aren’t any better, because we don’t have one (potentially) long running task. Instead we have an unknown number of potentially long running tasks which may occur at any time. You can’t really prepare for that in the same way.
This is where asnyc I/O comes into play. We don’t create a separate processing thread like we do with an async Servlet. Instead, we create a special listener class on either the request’s input buffer or the response’s output buffer. The listener instructs the Servlet on how to read or write data when the I/O is available. When it isn’t available, the Servlet won’t block. Instead, (and I’m 90% sure on this) the server will reallocate its worker thread to handle other requests. When the I/O is ready again, the thread is given back to the Servlet, and processing continues until either the read/write operation is complete, or the I/O becomes unavailable again.
Servlet 3.1’s implementation of async I/O is not quite perfect. There’s a lot of gotchas, and it is easy to screw up. I’m going to give you an extremely basic example of how it works, as well as a collection of links containing more information. I highly recommend you check these links out if you’re interested in using this feature.
Here’s the example:
package net.cmw;
import javax.servlet.*;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
/**
* Basic example of async I/O in a Servlet
*/
@WebServlet(name = "basic-async-io",
urlPatterns = {"/basic-async-io"},
description = "Basic Async IO Example",
asyncSupported = true) // notice the asyncSupported property`
public class BasicAsyncIOServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException,
IOException {
resp.addHeader("Content-Type", "text/plain");
final AsyncContext async = req.startAsync();
// We have to retrieve the response's stream as an ServletOutputStream
final ServletOutputStream output = resp.getOutputStream();
// We set our Listener class on the output stream
output.setWriteListener(new BasicWriteListener(output, async));
}
}
/**
* A basic implementation of the WriteListener interface
*/
class BasicWriteListener implements WriteListener {
private ServletOutputStream output;
private AsyncContext async;
public BasicWriteListener(ServletOutputStream output, AsyncContext async) {
this.output = output;
this.async = async;
}
@Override
public void onWritePossible() throws IOException {
if (output.isReady()) {
output.print("Some content");
}
// We still call this just like in an async Servlet. Only call it
// once writing (or reading) is finished.
async.complete();
}
@Override
public void onError(Throwable t) {
System.out.println(t.getMessage());
async.complete();
}
}There it is. The onWritePossible method (and onReadPossible for a ReadListener) is where things can get tricky (the example I wrote above is so simple as to be useless for practical purposes). Now here those links I mentioned:
- http://www.jayway.com/2014/05/16/async-servlets/
- http://www.javacodegeeks.com/2015/04/on-servlets-and-async-servlets.html
- http://www.slideshare.net/SimoneBordet/servlet-31-async-io
Study up before using this feature.
When to Use this Feature?
I was talking to a colleague about asnyc Servlet I/O, and he asked whether there was really a need for most applications to use such a feature to free up server threads. It’s a fair question, and for many applications, there likely is no need. However, if you find your app handling thousands, or even hundreds of thousands of requests at once, it might be worth considering. Just be sure you’re careful in how you implement the feature (this goes for regular async Servlets as well). I’ve seen some claims from developers that using asynchronous processing and/or I/O led to tremendous performance improvements in high traffic applications. I’ve even seen reports that their well written async Java code outperformed node.js. Whether such claims are true is not the point. The point is that these features can be very effective in practice.