[英]Importing a Json file using script task in SSIS
I am new to importing API's using script task in SSIS.我是在 SSIS 中使用脚本任务导入 API 的新手。 I have to manage the import of a straightforward API JSON file but this second API JSON file is a bit tricky.
我必须管理一个简单的 API JSON 文件的导入,但第二个 API Z0ECD11C1D7A287401F8D14A 文件有点棘手。 I have been looking at the code and just don't know what I'm doing wrong.
我一直在看代码,只是不知道我做错了什么。
My JSON file has a header which I need to determine how many pages of the API I need to loop through to get the data, but where I'm lost is, how do you import the next bit of data.我的 JSON 文件有一个 header 我需要确定 API 的多少页我需要循环获取数据,但是我丢失的下一位数据是如何导入的, I think I am close but since I'm new to this I could do it with a bit of guidance.
我想我很接近,但由于我是新手,所以我可以在一些指导下做到这一点。
My code in the script task below, the have seen the stream reader import the data but I do not what use (eg List, class or dictionary)to import the countIn
, countOut
, etc columns.我在下面的脚本任务中的代码,已经看到 stream 阅读器导入数据,但我不使用什么(例如 List、class 或字典)来导入
countIn
、 countOut
等列。
try
{
//Call getWebServiceResult to return our WorkGroupMetric array
WorkGroupMetric[] outPutMetrics = GetWebServiceResult(wUrl);
//For each group of metrics output records
foreach (var metric in outPutMetrics)
{
APIBuffer.AddRow();
APIBuffer.count = metric.count;
APIBuffer.currentpage = metric.currentpage;
APIBuffer.totalpages = metric.totalpages;
APIBuffer.countIn = metric.result.countIn;
APIBuffer.countOut = metric.result.countOut;
APIBuffer.type = metric.result.type;
APIBuffer.countLine = metric.result.countLine;
APIBuffer.from = metric.result.from;
APIBuffer.to = metric.result.to;
}
}
catch (Exception e)
{
FailComponent(e.ToString());
}
}
private WorkGroupMetric[] GetWebServiceResult(string wUrl)
{
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
WorkGroupMetric[] jsonResponse = null;
try
{
//Test the connection
if (httpWResp.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = httpWResp.GetResponseStream();
string jsonString = null;
//Set jsonString using a stream reader
using (StreamReader reader = new StreamReader(responseStream))
{
jsonString = reader.ReadToEnd().Replace("\\", "");
reader.Close();
}
//Deserialize our JSON
JavaScriptSerializer sr = new JavaScriptSerializer();
//JSON string comes in with a leading and trailing " that need to be removed for parsing to work correctly
//The JSON here is serialized weird, normally you would not need this trim
jsonResponse = sr.Deserialize<WorkGroupMetric[]>(jsonString.Trim('"'));
}
//Output connection error message
else
{
FailComponent(httpWResp.StatusCode.ToString());
}
}
//Output JSON parsing error
catch (Exception e)
{
FailComponent(e.ToString());
}
return jsonResponse;
throw new NotImplementedException();
}
private void FailComponent(string errorMsg)
{
bool fail = false;
IDTSComponentMetaData100 compMetadata = this.ComponentMetaData;
compMetadata.FireError(1, "Error Getting Data From Webservice!", errorMsg, "", 0, out fail);
}
}
//Class to hold our workgroup metrics
class WorkGroupMetric
{
public string count { get; set; }
public string currentpage { get; set; }
public string totalpages { get; set; }
public List<Result> result { get; set; }
}
class Result
{
public string countIn { get; set; }
public string countOut { get; set; }
public string type { get; set; }
public string countLine { get; set; }
public string from { get; set; }
public string to { get; set; }
}
So I have got the code working, posting it so t can help someone in the future.所以我已经让代码正常工作,发布它以便将来可以帮助某人。
try
{
//Call getWebServiceResult to return our WorkGroupMetric array
WorkGroupMetric outPutMetrics = GetWebServiceResult(wUrl);
//For each group of metrics output records
//foreach (var metric in outPutMetrics)
//{
var ts = outPutMetrics.results;
totalcount = Int32.Parse(outPutMetrics.count);
foreach (var a in ts)
{
Output0Buffer.AddRow();
//Output0Buffer.count = outPutMetrics.count;
Output0Buffer.countIn = a.countIn;
Output0Buffer.countOut = a.countOut;
Output0Buffer.type = a.type;
Output0Buffer.countLine = a.countLine;
Output0Buffer.from = a.from;
Output0Buffer.to = a.to;
i = i + 1;
}
Console.Write(i);
}
catch (Exception e)
{
FailComponent(e.ToString());
}
}
private WorkGroupMetric GetWebServiceResult(string wUrl)
{
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
WorkGroupMetric jsonResponse = null;
try
{
//Test the connection
if (httpWResp.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = httpWResp.GetResponseStream();
string jsonString = null;
//Set jsonString using a stream reader
using (StreamReader reader = new StreamReader(responseStream))
{
jsonString = reader.ReadToEnd().Replace("\\", "");
reader.Close();
}
//Deserialize our JSON
JavaScriptSerializer sr = new JavaScriptSerializer();
//JSON string comes in with a leading and trailing " that need to be removed for parsing to work correctly
//The JSON here is serialized weird, normally you would not need this trim
jsonResponse = sr.Deserialize<WorkGroupMetric>(jsonString.Trim('"'));
}
//Output connection error message
else
{
FailComponent(httpWResp.StatusCode.ToString());
}
}
//Output JSON parsing error
catch (Exception e)
{
FailComponent(e.ToString());
}
return jsonResponse;
throw new NotImplementedException();
}
private void FailComponent(string errorMsg)
{
bool fail = false;
IDTSComponentMetaData100 compMetadata = this.ComponentMetaData;
compMetadata.FireError(1, "Error Getting Data From Webservice!", errorMsg, "", 0, out fail);
}
}
//Class to hold our work group metrics
class WorkGroupMetric
{
public string count { get; set; }
public string currentpage { get; set; }
public string totalpages { get; set; }
public Result[] results { get; set; }
}
class Result
{
public string countIn { get; set; }
public string countOut { get; set; }
public string type { get; set; }
public string countLine `enter code here`{ get; set; }
public string from { get; set; }
public string to { get; set; }
}
Going by this example通过这个例子
https://www.tomasvera.com/programming/using-javascriptserializer-to-parse-json-objects/ https://www.tomasvera.com/programming/using-javascriptserializer-to-parse-json-objects/
Possibly this should be your C# class definition, using an array not a List
可能这应该是您的 C# class 定义,使用数组而不是
List
class WorkGroupMetric
{
public string count { get; set; }
public string currentpage { get; set; }
public string totalpages { get; set; }
public Result[] results { get; set; }
}
class Result
{
public string countIn { get; set; }
public string countOut { get; set; }
public string type { get; set; }
public string countLine { get; set; }
public string from { get; set; }
public string to { get; set; }
}
Also you're only getting one WorkGroupMetric per call, not an array of them so you should probably be using this:此外,您每次调用只会获得一个 WorkGroupMetric,而不是它们的数组,因此您可能应该使用它:
WorkGroupMetric
not this不是这个
WorkGroupMetric[]
throughout始终
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.