簡體   English   中英

janusgraph加載記錄期間提交性能低下

[英]Low commit performance during janusgraph load records

當我試圖從MySQL加載的數據量,我提交每個記錄與卡桑德拉后端和elasticsearch用於構建索引JanusGraph, 采用8螺紋 ;

一開始,程序將以280條記錄/秒的速度加載;
但是當它處理幾秒鍾時下降到1〜10條記錄/秒 ;

我嘗試修改緩沖區大小頁面大小塊大小更新百分比等配置,但是並沒有明顯改善。

我只是在徘徊我是否想念什么以及造成這種情況的原因...

后面的代碼是My commit過程, dataMap是fastJson對象, g是janusgraph遍歷源;

    Long countryId = dataMap.getLong("countryId");

    Long uid = dataMap.getLong("uid");
    String phoneNum = dataMap.getString("phoneNumber");
    String fbId = dataMap.getString("fbId");
    Long createTime = dataMap.getLong("createTime");

    if (uid == null) {
        return;
    }
    Vertex uidVertex = g.addV("uid").next();
    uidVertex.property("uid_code", uid);

    if (createTime != null)
        uidVertex.property("create_time", createTime);
    if (status != null)
        uidVertex.property("status", status);

    g.tx().commit();

    if (phoneNum != null) {
         Vertex phoneVertex = KfkMsgParser.createMerge(g, "phone", "phone_num", phoneNum);

        Edge selfPhone = uidVertex.addEdge("user_phone", phoneVertex);
        selfPhone.property("create_time", bind.of("create_time", dataMap.getLong("createTime")));
        selfPhone.property("uid_code", bind.of("uid_code", uid));
        selfPhone.property("phone_num", bind.of("phone_num", phoneNum));
        g.tx().commit();
    }

    if(fbId != null){
        long endTamp2 = System.currentTimeMillis();
        Vertex fbVertext = KfkMsgParser.createMerge(g, "fb_id", "fb_account",fbId);

        Edge selfFb = uidVertex.addEdge("user_fb",fbVertext);
        if (createTime != null)
            selfFb.property("create_time",bind.of("create_time",createTime));
        g.tx().commit();
    }

這是createMerge功能:

private static Vertex createMerge(GraphTraversalSource g, String label, String propertyKey, Object propertyValue) {
    Optional<Vertex> vertexOptional = g.V().hasLabel(label).has(propertyKey, propertyValue).tryNext();
    if (vertexOptional.isPresent()) {
        return vertexOptional.get();
    }
    Vertex vertex = g.addV(label).next();
    vertex.property(propertyKey, propertyValue);
    return vertex;
}

建立索引時出問題了。
我在Google網上論壇中找到了這樣一個主題: https//groups.google.com/forum/#! msg / janusgraph-users / VPIUdlC4wNo / KiHM-s2aAwAJ
並且知道獲得2000〜3000條記錄/秒。

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM