[英]Multithreading for making http post requests to web service
I want to send multiple HTTP post requests to a Web Service in C# .For example , if n=3 then http post requests from 3 xml files should be made and also the response should be written in a file.Once the first 3 requests are made then the next 3 requests will be made . 我想在C#中向Web服务发送多个HTTP post请求。例如,如果n = 3,则应该发出来自3个xml文件的http post请求,并且响应应该写在文件中。前3个请求是然后做出接下来的3个请求。 So i made the following code but i was getting random outputs at first. 所以我做了以下代码,但我首先得到随机输出。 But now i am getting either out of index range exception in the inner for loop or Internal server error (500). 但现在我要么在内部for循环或内部服务器错误(500)中超出索引范围异常。 Plz suggest appropriate changes. Plz建议适当的改变。 I am using .NET4.0 我使用的是.NET4.0
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Threading;
using System.Xml;
using System.Net;
using System.Threading.Tasks;
namespace ConsoleApplication5
{
class Program
{
static void Main(string[] args)
{
int n = 0;
Console.WriteLine("Enter the number");
string s = Console.ReadLine();
int.TryParse(s, out n);
string path = "C:\\";
string[] files = null;
files = Directory.GetFiles(path, "*.xml", SearchOption.TopDirectoryOnly);
List<Task> tasks = new List<Task>(files.Length);
for (int i = 0; i < files.Length; i += n)
{
for (int j = 0; j < n; j++)
{
int x = i + j;
if (x < files.Length && files[x] != null)
{
Task t = new Task(() => function(files[x]));
t.Start();
tasks.Add(t);
}
}
if (tasks.Count > 0)
{
Task.WaitAll(tasks.ToArray(), Timeout.Infinite); // or less than infinite
tasks.Clear();
}
}
}
public static void function(string temp)
{
XmlDocument doc = new XmlDocument();
doc.Load(temp);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://10.76.22.135/wpaADws/ADService.asmx");
request.ContentType = "text/xml;charset=\"utf-8\"";
request.Accept = "text/xml";
request.Method = "POST";
Stream stream = request.GetRequestStream();
doc.Save(stream);
stream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (StreamReader rd = new StreamReader(response.GetResponseStream()))
{
string soapResult = rd.ReadToEnd();
doc.LoadXml(soapResult);
File.WriteAllText(temp, doc.DocumentElement.InnerText);
//XmlTextWriter xml=new XmlTextWriter(
Console.WriteLine(soapResult);
Console.ReadKey();
}
}
}
} }
This code works . 这段代码有效。 Explaination : 解释:
Directory.getFiles() helps us to get the .xml files in the string array . Directory.getFiles()帮助我们获取字符串数组中的.xml文件。 (we have to pass .xml as a parameter) . (我们必须传递.xml作为参数)。
SO now what basically happens is for each file we get at the source pat , a thread is created . 那么现在基本上发生的是我们从源码获取的每个文件,创建一个线程。
And after a request is made to the web service , we get the response by getResponse() and the response is written in .xml files which are stored at the destination paths. 在向Web服务发出请求后,我们通过getResponse()获取响应,并将响应写入存储在目标路径的.xml文件中。
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.IO; using System.Threading; using System.Xml; using System.Net; namespace ConsoleApplication4 { class Program { int flag = 1; string destination; string source; static void Main(string[] args) { Console.ForegroundColor = ConsoleColor.Red; Console.WriteLine("**************************** Send HTTP Post Requests **************************"); int n = 0; Program p = new Program(); Console.WriteLine("Enter the number of requests you want to send at a time"); string s = Console.ReadLine(); int.TryParse(s, out n); Console.WriteLine("Enter Source"); p.source = Console.ReadLine(); Console.WriteLine("Enter Destination"); p.destination = Console.ReadLine(); string[] files = null; files = Directory.GetFiles(p.source, "*.xml", SearchOption.TopDirectoryOnly); Thread[] thread = new Thread[files.Length]; int len = files.Length; for (int i = 0; i<len; i+=n) { int x = i; //Thread.Sleep(5000); for (int j = 0; j < n && x < len; j++) { var localx = x; thread[x] = new Thread(() => function(files[localx], p)); thread[x].Start(); Thread.Sleep(50); //thread[x].Join(); x++; } int y = x - n; for (; y < x; y++) { int t = y; thread[t].Join(); } } // thread[0] = new Thread(() => function(files[0])); //thread[0].Start(); Console.ReadKey(); } public static void function(string temp,Program p) { XmlDocument doc = new XmlDocument(); doc.Load(temp); string final_d=p.destination + "response " + p.flag + ".xml"; p.flag++; HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://10.76.22.135/wpaADws/ADService.asmx"); request.ContentType = "text/xml;charset=\\"utf-8\\""; request.Accept = "text/xml"; request.Method = "POST"; Stream stream = request.GetRequestStream(); doc.Save(stream); stream.Close(); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); using (StreamReader rd = new StreamReader(response.GetResponseStream())) { string soapResult = rd.ReadToEnd(); doc.LoadXml(soapResult); File.WriteAllText(final_d, doc.DocumentElement.InnerText); //XmlTextWriter xml=new XmlTextWriter( Console.WriteLine(soapResult); //Console.ReadKey(); } }
} } }}
The IndexOutOfRangeException
you experienced in your original post was due to the improper index handling on the last batch of files you were processing. 您在原始帖子中遇到的IndexOutOfRangeException
是由于您正在处理的最后一批文件的索引处理不当造成的。 That last batch can be incomplete and you treated that as a regular batch of set size 最后一批可能不完整,您将其视为常规批量设置
(n=3 in your post) (你的帖子中n = 3)
Since you're moving to TPL and Tasks
, I suggest Parallel Programming with Microsoft .NET , and the pipeline pattern which seems very appropriate to your scenario. 由于您正在转向TPL和Tasks
,我建议使用Microsoft .NET进行并行编程 ,以及看起来非常适合您的场景的管道模式 。 You can harness the power of concurrent collections and the producer/consumer pattern together with the pipeline, like below. 您可以利用并发集合的强大功能和生产者/消费者模式以及管道,如下所示。 BlockingCollection ensures concurrent adding of items and the BlockingCollection.GetConsumingEnumerable call produces a consuming blocking enumerator for your collection. BlockingCollection确保并发添加项目,并且BlockingCollection.GetConsumingEnumerable调用为您的集合生成消耗阻塞枚举器。
const int BUFFER_SIZE = 3; // no concurrent items to process
const string XML_FOLDER_PATH = "<whatever>";
public static void Pipeline()
{
var bufferXmlFileNames = new BlockingCollection<string>(BUFFER_SIZE);
var bufferInputXmlDocuments = new BlockingCollection<XmlDocument>(BUFFER_SIZE);
var bufferWebRequests = new BlockingCollection<HttpWebRequest>(BUFFER_SIZE);
var bufferSoapResults = new BlockingCollection<string>(BUFFER_SIZE);
var f = new TaskFactory(TaskCreationOptions.LongRunning, TaskContinuationOptions.None);
// Stage 1: get xml file paths
var stage1 = f.StartNew(() => {
try
{
foreach (var phrase in Directory.GetFiles(XML_FOLDER_PATH, "*.xml", SearchOption.TopDirectoryOnly))
{ // build concurrent collection
bufferXmlFileNames.Add(phrase);
}
}
finally
{ // no more additions acceptedin
bufferXmlFileNames.CompleteAdding();
}
});
// Stage 2: ProduceInputXmlDocuments(bufferXmlFileNames, bufferInputXmlDocuments)
var stage2 = f.StartNew(() => {
try
{
foreach (var xmlFileName in bufferXmlFileNames.GetConsumingEnumerable())
{
XmlDocument doc = new XmlDocument();
doc.Load(xmlFileName);
bufferInputXmlDocuments.Add(doc);
}
}
finally
{
bufferInputXmlDocuments.CompleteAdding();
}
});
// Stage 3: PostRequests(BlockingCollection<XmlDocument> xmlDocs, BlockingCollection<HttpWebRequest> posts)
var stage3 = f.StartNew(() => {
try
{
foreach (var xmlDoc in bufferInputXmlDocuments.GetConsumingEnumerable())
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://10.76.22.135/wpaADws/ADService.asmx");
request.ContentType = "text/xml;charset=\"utf-8\"";
request.Accept = "text/xml";
request.Method = "POST";
//
Stream stream = request.GetRequestStream();
xmlDoc.Save(stream);
stream.Close();
//
bufferWebRequests.Add(request);
}
}
finally
{
bufferWebRequests.CompleteAdding();
}
});
// Stage 4: ProcessResponses(bufferWebRequests, bufferSoapResults)
var stage4 = f.StartNew(() =>
{
try
{
foreach (var postRequest in bufferWebRequests.GetConsumingEnumerable())
{
HttpWebResponse response = (HttpWebResponse)postRequest.GetResponse();
using (StreamReader rd = new StreamReader(response.GetResponseStream()))
{
string soapResult = rd.ReadToEnd();
bufferSoapResults.Add(soapResult);
}
}
}
finally
{
bufferSoapResults.CompleteAdding();
}
});
// stage 5: update UI
var stage5 = f.StartNew(() =>
{
foreach (var soapResult in bufferSoapResults.GetConsumingEnumerable())
{
Console.WriteLine(soapResult);
}
});
// display blocking collection load state,
// the number of elements in each blocking collection of the pipeline stages
// you can supress this call completely, because it is informational only
var stageDisplay = f.StartNew(
() =>
{
while (true)
{
Console.WriteLine("{0,10} {1,10} {2,10} {3,10}", bufferXmlFileNames.Count, bufferInputXmlDocuments.Count, bufferWebRequests.Count, bufferSoapResults.Count);
//check last stage completion
if (stage5.IsCompleted)
return;
}
}
);
Task.WaitAll(stage1, stage2, stage3, stage4, stage5); //or
//Task.WaitAll(stage1, stage2, stage3, stage4, stage5, stageDisplay);
}
How about using tasks like this: 如何使用这样的任务:
List<Task> tasks = new List<Task>(n);
for (int i = 0; i < files.Length; i += n)
{
for (int j = 0; j < n; j++)
{
int x = i + j;
if (x < files.Length && files[x] != null)
{
Task t = new Task(() => function(files[x]));
t.Start();
tasks.Add(t);
}
}
if (tasks.Count > 0)
{
Task.WaitAll(tasks.ToArray(), Timeout.Infinite); // or less than infinite
tasks.Clear();
}
}
I tried to be a little tidier on the indexing... 我试着在索引方面稍微整理一下......
Also, note that the int x = i + j;
另外,请注意int x = i + j;
in the inner loop is important due to how C# captures variables for the lambda. 因为C#如何捕获lambda的变量,所以在内循环中很重要。
If the problem is tracing down indexing arithmetic, maybe use indexing variables with meaningful names? 如果问题是追溯索引算法,可能使用具有有意义名称的索引变量?
List<Task> tasks = new List<Task>(taskCount);
for (int filesIdx = 0; filesIdx < files.Length; filesIdx += taskCount)
{
for (int tasksIdx = 0; tasksIdx < taskCount; tasksIdx++)
{
int index = filesIdx + tasksIdx;
if (index < files.Length && files[index] != null)
{
Task task = new Task(() => function(files[index]));
task.Start();
tasks.Add(task);
}
}
if (tasks.Count > 0)
{
Task.WaitAll(tasks.ToArray(), Timeout.Infinite); // or less than infinite
tasks.Clear();
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.