Streaming with New .NET HttpClient and HttpCompletionOption.ResponseHeadersRead
How to consume a streaming endpoint with new .NET System.Net.Http.HttpClient and the role of HttpCompletionOption.ResponseHeadersRead
1 August 2012
3 minutes read
Related Posts
For a while now, I have been playing with a new application I created for fun: TweetMapR and I am not sure where it is going :) The application itself consumes Twitter Streaming API and broadcasts the retrieved data to the connected clients through SignalR. I used new HttpClient to connect to Twitter Streaming API and I used OAuth as authentication protocol. In fact, I created my own OAuth Twitter client to connect to Twitter which is so raw right now but works well.
But how to send a request to Twitter Streaming API and keep it open infinitely was my main question for the whole time. So, I brought up the ILSpy and decompiled the System.Net.Http.dll to see what is really going on and I realized that it was going to be so easy. First of all, if your request is a GET request, you are covered by the framework. The GetStreamAsync method of the HttpClient sends a GET request and returns back the stream as soon as it completes reading the response headers. The following example code shows the usage.
using (HttpClient httpClient = new HttpClient()) { httpClient.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite); var requestUri = "http://localhost:6797"; var stream = httpClient.GetStreamAsync(requestUri).Result; using (var reader = new StreamReader(stream)) { while (!reader.EndOfStream) { //We are ready to read the stream var currentLine = reader.ReadLine(); } } }
First of all, don’t use Result on Task :) It blocks but I used it here for the sake of simplicity to stick with the main point of this post :) Assuming that the localhost:6797 is a streaming endpoint here, the above code should work perfectly fine and shouldn’t be timed out as we set the Timeout to System.Threading.Timeout.Infinite. But what if the request we need to send is a POST request? In that case, we have a little more work to do here.
If we just try to send a POST request with the PostAsync method or any of its variants, the response will never come back unless the server ends the request because it will try to read the response till the end. In order to omit this problem, we can pass a HttpCompletionOption enumuration value to specify the completion option. The HttpCompletionOption enumeration type has two members and one of them is ResponseHeadersRead which tells the HttpClient to only read the headers and then return back the result immediately. The following code shows a sample example where we need to send a form-urlencoded POST request to a streaming endpoint.
using (HttpClient httpClient = new HttpClient()) { httpClient.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite); var requestUri = "http://localhost:6797"; var formUrlEncodedContent = new FormUrlEncodedContent( new List<KeyValuePair<string, string>>() { new KeyValuePair<string, string>("userId", "1000") }); formUrlEncodedContent.Headers.ContentType = new MediaTypeHeaderValue("application/x-www-form-urlencoded"); var request = new HttpRequestMessage(HttpMethod.Post, requestUri); request.Content = formUrlEncodedContent; var response = httpClient.SendAsync( request, HttpCompletionOption.ResponseHeadersRead).Result; var stream = response.Content.ReadAsStreamAsync().Result; using (var reader = new StreamReader(stream)) { while (!reader.EndOfStream) { //We are ready to read the stream var currentLine = reader.ReadLine(); } } }
Again, don’t use Result as I do here :) We have obviously more noise this time but if this is something that you will use often, there is nothing stopping you to write an extension method.
Hey, I'm Tugberk 👋
Coder 👨🏻💻, Speaker 🗣, Author 📚, ex-Microsoft MVP 🕸, Blogger 💻
I live in London, UK 🏡
