繁体   English   中英

在 DataWeave 中递归遍历和展平 JSON 对象

[英]Recursivly traverse and flatten JSON object in DataWeave

我想遍历并展平一个具有以下结构的大型 JSON 文件,显示产品层次结构(将其视为在线商店中的导航):

productGroups: [
    {
        "key": "child 1"
        ...
        "childrenProductGroups": [
            {
                "key": "child 1.1",
                ...,
                "childrenProductGroups": []
            },
            {
                "key": "child 1.2"
                ...
                "childrenProductGroups": [
                    {
                        "key": "child 1.2.1",
                        ...,
                        "childrenProductGroups": [
                            {
                                "key": "child 1.2.1.1",
                                ...,
                                childrenProductGroups": [
                                    ...
                                ]
                            }
                        ]
                    },
                    {
                        "key": "child 1.2.2",
                        ...,
                        "childrenProductGroups": []
                    }
                ]
                
            },
            {
                "key": "child 1.3",
                ...,
                "childrenProductGroups": [
                    ...
                ]
            }
        ]
    }, 
    {
        "key": "child 2",
        ...,
        "childrenProductGroups": [
            ...
        ]
    },
    {
        "key": "child 3",
        ...,
        "childrenProductGroups": [
            ...
        ]
    }
]

我想以这样的格式将它们展平:

{
    "hierarchieSet": [
        {
            "Nodeid": "00000001", # Number in this json
            "Nodename": "child 1",
            "Tlevel": "01", # First child of product group
            "Parentid": "00000000", # Parent is null
            "Childid": "00000002", # Child node number
            "Nextid": "00000008" # Node number on the same level (child 2)
        }, 
        {
            "Nodeid": "00000002",
            "Nodename": "child 1.1",
            "Tlevel": "02",
            "Parentid": "00000001",
            "Childid": "00000003",
            "Nextid": "00000003"
        }, 
        {
            "Nodeid": "00000003",
            "Nodename": "child 1.2",
            "Tlevel": "02",
            "Parentid": "00000002",
            "Childid": "00000005",
            "Nextid": "00000007"
        }, 
        {
            "Nodeid": "00000004",
            "Nodename": "child 1.2.1",
            "Tlevel": "03",
            "Parentid": "00000003",
            "Childid": "0000005",
            "Nextid": "00000006"
        }
        , 
        {
            "Nodeid": "00000005",
            "Nodename": "child 1.2.1.1",
            "Tlevel": "04",
            "Parentid": "00000004",
            "Childid": "0000000", #No more children
            "Nextid": "00000000"
        }, 
        {
            "Nodeid": "00000006",
            "Nodename": "child 1.2.2",
            "Tlevel": "03",
            "Parentid": "00000003",
            "Childid": "0000000",
            "Nextid": "00000000"
        }, 
        {
            "Nodeid": "00000007",
            "Nodename": "child 1.3",
            "Tlevel": "02",
            "Parentid": "00000001",
            "Childid": "0000000",
            "Nextid": "00000000"
        }, 
        {
            "Nodeid": "00000008",
            "Nodename": "child 2",
            "Tlevel": "01",
            "Parentid": "00000000",
            "Childid": "0000009", # 00000009 not shown
            "Nextid": "00000014" # 
        }, 
        ...
        {
            "Nodeid": "000000014",
            "Nodename": "child 3",
            "Tlevel": "01",
            "Parentid": "00000000",
            "Childid": "00000015",
            "Nextid": "00000000" # 00000010 does not exist
        }
    ]
}

因此,我确定了一些主要问题:

  • 树结构的递归
  • 变换元素
  • 扁平化结构
  • 跟踪父母、兄弟姐妹和孩子
  • 跟踪递归级别
  • 格式化数字

我试图通过两种不同的方法解决这个问题:

  • 使用 DataWeave 转换所有元素
  • 使用Java遍历结构

由于我对函数式编程还很陌生,因此我更加关注 Java 实现,但遇到了许多问题。

Java方法
Java流程
读取 json > Init Tree var 并为顶层数组中的每个元素分配 Java 实例 > 在Tree.java调用traverse(data, level)
树.java:

import org.json.JSONObject;

public class Tree {
    private int id = 0;
    private List<Node> nodes = new ArrayList<Node>();
    
    public Tree() {
        nodes.add(new Node("01", "00000001", "HOME", "01", "00000000", "00000002", "00000000"));
    }
    
    public void traverse(String data, int level) {
        System.out.println(data);
        // TODO parse json
    }
    
    private void visit(JSONObject parent, JSONObject node, int level) {
        id++;
        nodes.add(new Node("01", String.valueOf(id), node.getString("key"), String.valueOf(level), "", "", ""));
    }
    
    public List<Node> getNodes() {
        return nodes;
    }

    private static class Node {
        private String zshop, nodename, parentid, childid, nextid, nodeid, tlevel;
        
        public Node(String zshop, String nodeid, String nodename, String tlevel, String parentid, String childid, String nextid) {
            this.zshop = zshop;
            this.nodeid = nodeid;
            this.nodename = nodename;
            this.tlevel = tlevel;
            this.parentid = parentid;
            this.childid = childid;
            this.nextid = nextid;
        }
    }
}

调用 invoke 操作时,我使用此有效负载:

%dw 2.0
output application/java
---
{
    data: vars.rootMessage.payload as String,
    level: 1
}

但这会产生以下错误:

“无法强制对象{编码:UTF-8,媒体类型:文本/json;字符集=UTF-8,mimeType:文本/json,原始:org.mule.weave.v2.el.SeekableCursorStream@50ecee52}(org.mule。 weave.v2.el.MuleTypedValue@511ba9cc) 到字符串

5| 数据:vars.rootMessage.payload as String,^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 跟踪:在主(行: 5, column: 7)" 评估表达式: "%dw 2.0 output application/java --- { data: vars.rootMessage.payload as String, level: 1 }"。

我尝试了很多事情:

  • 将其转换为我用 Java 编写的ProductGroup对象
  • 尝试将检索到的对象org.json.JSONObjectorg.json.JSONObject
  • 尝试缓冲和读取vars.rootMessage.payload (Binary)

但是我无法用这些问题中的任何一个来解决它。

DataWeave 方法我的 .dw 脚本

%dw 2.0
fun append
(item: Object, acc: Object = {
}) = acc ++ item

fun mapper(item: Object) = 
{
    Zshop: "01",
    Nodeid: "00000000",
    Nodename: item.key as Number as String {format: ""},
    Tlevel: "02",
    Parentid: "00000000",
    Childid: "00000000",
    Nextid: "00000000"
}
    
fun traverse(a: Array, level: Number) = 
    a map $ flatMap(value, index) -> value
    
output application/json
---
{
    test: payload.productGroups reduce (item, acc) -> append(mapper(item), acc)
}

我试图解决一些问题的地方。 mapper(item)应该创建 json 对象,我可以使用appender(item, acc)将其附加到最终输出。 递归已经被勾勒出来,但还不是我的主要关注点。

这产生了这个结果:

(original payload),
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "800",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "110",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "720",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "710",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
...

我想知道为什么我在没有任何对象结构的情况下得到平坦的结果。

我的问题:

  • Java:为什么我不能转换字符串或者它是如何正确完成的
  • DataWeave:有没有我看不到的简单解决方案?
  • 为什么它是一个平面结果而不是一个对象?
  • 对此, reduceflatMap函数的用法是否正确?

欢迎任何帮助和/或反馈。

Java:为什么我不能转换字符串或者它是如何正确完成的

JSON 不是字符串。 使用 write(payload,'application/json') 来获得字符串。

DataWeave:有没有我看不到的简单解决方案?

只需传递对象,它是Java中的Map。 因为它是树 - 每个分支都是这个 Map 中的另一个 Map。

为什么它是一个平面结果而不是一个对象?

它始终是对象。 Java 世界中没有其他东西。

对此,reduce 和 flatMap 函数的用法是否正确?

不。 mapObject 和递归应该是很好的方法。

根据我收到的帮助,我可以制定这个解决方案:

public class Tree implements Serializable {
    private int id = 0;
    private List<Node> nodes = new ArrayList<Node>();
    
    public Tree() {
        nodes.add(new Node("01", "00000001", "HOME", "0", "00000000", "00000002", "00000000"));
    }
    
    public void enter(String jsonString, Integer level) {
        JSONObject json = new JSONObject(jsonString);
        traverse(json, level);
    }
    
    public void traverse(JSONObject json, int level) {
        visit(json, level);
        
        JSONArray arr = json.getJSONArray("childProductGroups");
        if(arr != null) {
            for(int i = 0; i < arr.length(); i++) {
                traverse(arr.getJSONObject(i), level + 1);
            }
        }       
    }
    
    private void visit(JSONObject object, int level) {
        id++;
        nodes.add(new Node("01", String.valueOf(id), object.getString("name_de"), String.valueOf(level), "", "", ""));
    }
    
    public Node[] getNodes() {
        assignParentIds();
        assignNextIds();
        assignChildIds();
            
        Node[] nodeArr = new Node[nodes.size()];
        for(int i = 0; i < nodes.size(); i++) {
            nodeArr[i] = nodes.get(i);
        }
        return nodeArr;
    }
    
    private void assignParentIds() {
        Map<String, Node> lastNodesWithHigherLevel = new HashMap<String, Node>();
        for(Node node : nodes) {
            Node higher = lastNodesWithHigherLevel.get(String.valueOf(Integer.valueOf(node.tlevel) - 1));
            if(higher != null) {
                node.parentid = higher.nodeid;
            }   
            lastNodesWithHigherLevel.put(node.tlevel, node);
        }
    }
    
    private void assignNextIds() {
        Map<String, Node> lastNodeOnSameLevel = new HashMap<String, Node>();
        for(Node node : nodes) {
            Node last = lastNodeOnSameLevel.get(node.tlevel);
            if(last != null) {
                if(last.parentid.equals(node.parentid)) {
                    // If the last and this node have the same parent
                    last.nextid = node.nodeid;
                }
            }   
            lastNodeOnSameLevel.put(node.tlevel, node);
        }
    }

    private void assignChildIds() {
        Iterator<Node> parentIterator = nodes.iterator();
        Iterator<Node> childIterator = nodes.iterator();
        
        // Init child iterator one further
        if(childIterator.hasNext()) {
            childIterator.next();
        }
        
        do {
            Node parent = parentIterator.next();
            Node child = childIterator.next();
            // If level of parent is higher (<) set the value
            if(Integer.valueOf(parent.tlevel) < Integer.valueOf(child.tlevel)) {
                parent.childid = child.nodeid;
            }
        } while (childIterator.hasNext());
        // Because childIterator will first be done, but parentIterator might need one more reference, we need do while
    }
    
    private static class Node implements Serializable {
        public String zshop, nodename, parentid, childid, nextid, nodeid, tlevel;
        
        public Node(String zshop, String nodeid, String nodename, String tlevel, String parentid, String childid, String nextid) {
            this.zshop = zshop;
            this.nodeid = nodeid;
            this.nodename = nodename;
            this.tlevel = tlevel;
            this.parentid = parentid;
            this.childid = childid;
            this.nextid = nextid;
        }
        
        @Override
        public String toString() {
            return "{zshop:" + zshop + ", nodename:" + nodename + ", parentid:" + parentid + ", childid:" + childid
                    + ", nextid:" + nextid + ", nodeid:" + nodeid + ", tlevel:" + tlevel + "}";
        }
    }
}

和流程配置:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:java="http://www.mulesoft.org/schema/mule/java" xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
    xmlns:file="http://www.mulesoft.org/schema/mule/file"
    xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
http://www.mulesoft.org/schema/mule/java http://www.mulesoft.org/schema/mule/java/current/mule-java.xsd">
    <http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="e31dc867-3f9f-457b-8013-cd50a74c0af1" >
        <http:listener-connection host="0.0.0.0" port="8081" />
    </http:listener-config>
    <file:config name="File_Config" doc:name="File Config" doc:id="3279487e-61ca-4845-89a2-5332e7c02638" />
    <flow name="java-flow" doc:id="3c036218-560d-44f0-aef5-0c7f0ef4d776" >
        <http:listener doc:name="Listener" doc:id="265a1691-4f9b-40e2-a280-592daf719002" config-ref="HTTP_Listener_config" path="java">
            <http:response >
                <http:body ><![CDATA[#[output application/json ---
payload]]]></http:body>
            </http:response>
        </http:listener>
        <file:read doc:name="Copy_of_Read" doc:id="9f8b3c6f-f674-45f7-80fe-756f64602b30" config-ref="File_Config" path="response.json" />
        <set-variable doc:name="Set Variable" doc:id="ecff267b-3b73-45d1-924e-227a01a99e4e" variableName="Tree" value="#[null]" />
        <java:new doc:name="New Tree" doc:id="764d60bf-da41-4f17-8e13-c8bf31f141dc" class="valion.Tree" constructor="Tree()" target="Tree" />
        <foreach doc:name="For Each" doc:id="e612fd8b-636e-41c2-b603-e6861514306b" collection="#[payload.productGroups]">
            <java:invoke doc:name="Traverse" doc:id="96da4743-00d3-4970-a9e5-712877bcf2a9" class="valion.Tree" instance="#[vars.Tree]" method="enter(java.lang.String,java.lang.Integer)">
                <java:args><![CDATA[#[%dw 2.0
output application/java
---
{
    jsonString: write( payload,'application/json'),
    level: 1
}]]]></java:args>
            </java:invoke>
        </foreach>
        <java:invoke doc:name="Get Nodes" doc:id="999d5f4c-80b4-4793-9188-e17725ad030b" instance="#[vars.Tree]" class="valion.Tree" method="getNodes()">
        </java:invoke>
        <ee:transform doc:name="Transform Message" doc:id="b078bef3-45aa-4821-b7d1-720c2c7d0580">
            <ee:message>
                <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{
    hierarchieSet: payload map ( item , index ) -> {
        zshop: item.zshop,
        nodeid: item.nodeid,
        nodename: item.nodename,
        tlevel: item.tlevel,
        parentid: item.parentid,
        childid: item.childid,
        nextid: item.nextid
    }
}]]></ee:set-payload>
            </ee:message>
        </ee:transform>
    </flow>
</mule>

在此处输入图片说明

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM