简体   繁体   中英

Parse device data with simple json and building a comma delimited file in hdfs

My question is around how to parse a json response from a device's api. The json looks like below.

[{"name":"Device 1","label":"Switch State","value":"off"},{"name":"Device 2","label":"Switch State","value":"on"}]

public class Fubar{

public static void main(String[] args) throws Exception {
    boolean a = true;
    while (a) {
        Configuration conf = new Configuration();
        FileSystem hdfs = FileSystem.get(conf);
        String fileName = "filethingy" + "-" + new Date().getTime() + ".txt";

        boolean b = true;
        while(b){
            String connString = "url";
            URL url = new URL(connString);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("GET");
            BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
            String inputLine;
            out.writeBytes("TimeStamp"+",");
            out.writeBytes("DeviceName"+",");
            out.writeBytes("DeviceLabel"+",");
            out.writeBytes("DeviceStatus"+",");
            out.writeBytes("\n");
            while ((inputLine = in.readLine()) != null) {
                JSONParser jsonParser = new JSONParser();
                JSONObject jsonObject = (JSONObject) jsonParser.parse(inputLine);

                String date = new java.sql.Timestamp(System.currentTimeMillis()).toString();
                out.write(date.getBytes());
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("name"));
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("label"));
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("value"));
                //out.write(inputLine.getBytes());
                out.write("\n".getBytes());
                out.hflush();
            }

            in.close();
            Thread.sleep(30000);
            out.hflush();

        }
        out.close();
        hdfs.close();
        a=false;
        b=false;
    }

This doesn't work I know because I can't convert a JSONArray to a JSONObject. I'm having trouble figuring out how to pull apart the JSON correctly though to build ultimately a csv that looks like TimeStamp,DeviceName, DeviceLabel, DeviceStatus, 01012001, Device1, Switch Status, On. Someone's going to put a Hive table on top of the data when we're done. Any help would be appreciated.

Answered my own question.

while ((inputLine = in.readLine()) != null) {

                JSONParser jsonParser = new JSONParser();
                JSONArray jsonArray = (JSONArray) jsonParser.parse(inputLine);
                for (int i = 0; i < jsonArray.size(); i++) {
                JSONObject object = (JSONObject) jsonArray.get(i);  
                String date = new java.sql.Timestamp(System.currentTimeMillis()).toString();
                out.write(date.getBytes());
                out.write(",".getBytes());
                out.writeBytes((String) object.get("name"));
                out.write(",".getBytes());
                out.writeBytes((String) object.get("label"));
                out.write(",".getBytes());
                out.writeBytes(String.valueOf(object.get("value")));
                //out.write(inputLine.getBytes());
                out.write("\n".getBytes());
                out.hflush();

            }

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM