rita.hou | 2022-06-15 07:49:49 UTC | #1
After getting the url from the historical adherence request response, I'm not able to get the data from the url. When I post the data in the browser it works, the data shows in the html body. But when I use C# code to download the data, it comes back with unreadable data. Can someone help? Here's the code I'm using,
string urlAddress = "https://historical-adherence-prod-usw2-wemagents.s3.us-west-2.amazonaws.com/historicalAdherence/userDownloads/historicalAdherence/xxxxxxxxx
var httpClientHandler = new HttpClientHandler(); var httpClient = new HttpClient(httpClientHandler) { BaseAddress = new Uri(urlAddress) }; using (var response = httpClient.GetAsync(urlAddress)) { string responseBody = response.Result.Content.ReadAsStringAsync().Result;
}
brian.trezise | 2022-06-15 14:54:41 UTC | #2
When I post the data in the browser it works, the data shows in the html body
So if I'm understanding correctly, you're able to get the data from the returned URL from the notification by putting it in your browser, but your C# code isn't understanding the data returned by the URL, is that correct?
One thing that occurs to me is that the data comes back as gzip-compressed application/json data (which should be visible in the headers on the response from the GET). Your browser will automatically decompress the data and render the json whenever it sees those headers, but you'll have to decompress it yourself before attempting to read the data if you're doing this in code
brian.trezise | 2022-06-24 18:37:57 UTC | #3
the headers in question: content-encoding: gzip and content-type: application/json
I don't see anything in your code that is inflating the compressed file prior to attempting to read the data, so that's my suspicion as to the cause
Eos_Rios | 2022-06-24 18:37:57 UTC | #4
Expanding on this to confirm the problem.
The webclient native to dotnet doesn't do it by default. I use a derived class from Webclient and modify it;
protected override WebRequest GetWebRequest(Uri address) { _request = base.GetWebRequest(address);
//var httpRequest = _request as HttpWebRequest;
switch (_request ) { case HttpWebRequest httpRequest: httpRequest.AllowAutoRedirect = AutoRedirect; httpRequest.CookieContainer = CookieContainer; httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Setup?.Invoke(httpRequest); break; case FileWebRequest fileRequest: // Guess we do nothing? break;
} return _request; }
The AutomaticDecompression defaults to off/none in the underlying objects so one must either use a better object, or handle it manually.
rita.hou | 2022-06-16 13:33:25 UTC | #5
Thank you @brian.trezise and @Eos_Rios for the suggestions. I find it not easy to define http response header content encoding and content type tho, here's the code that I come up with and it works for me.
HttpClient _client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate });
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, urlAddress);
_client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
var result = await _client.SendAsync(request); var compressedStream = await result.Content.ReadAsStringAsync();
John_Carnell | 2022-06-24 18:38:02 UTC | #6
This post was migrated from the old Developer Forum.
ref: 15156