在 LUA 中使用字符串連接絕對可實現的最小 GC？

Question

運行時： lua 5.1.x 在 ARM64 下編譯，不允許使用 c-modules

示例代碼，准備運行： https : //paste.gg/p/anonymous/08f364480a5f470e9da610ab565e11c0

我需要在循環中每 X ms 連接一堆字符串。 根據我的理解，LUA 支持字符串實習，這意味着字符串文字被“緩存”而不是每次都分配。 因此，只有直接調用tostring() （或.. Sugar）才會分配。 現有字符串值的其余部分將通過引用傳遞。

到目前為止我所做的：

• 消除了所有整數-> 字符串分配（通過 LUT）
• 盡管tostring(bool)確實從緩存中返回了內部字符串，但我也消除了它
• 通過通過索引工作的表創建偽字符串構建器（每個約 16B）
• “預先調整大小”的表以避免關聯添加的成本，並使其成為全局表，因此不會每次都收集和重新創建
• 使用 table.concat() 進行最后的大字符串連接

最后的結果還是讓我很傷心：

Allocated pre-concat: 2.486328125 KB
Allocated post-concat: 39.7451171875 KB
Total table meta bytes: 1544 B
Total tostring meta bytes: 273 B

有什么我遺漏的還是我在這里達到了 LUA 的極限？

Answer 1

我假設您提到的問題與函數CONTAINER.PopulateState的內存消耗有關。 我認為你的代碼沒問題，但你沒有衡量正確的東西。 我刪除了所有collectgarbage以便將它們收集到代碼的一個部分中：

print("Allocated PRE-concat:                " ..  tostring(collectgarbage("count")))

-- First time
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

-- One more try
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

-- One more try
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

結果非常不同，而且更有意義：

Allocated PRE-concat:                48.70703125
Allocated POST-concat BEFORE-COLLECT:54.3232421875
Allocated POST-concat  AFTER-COLLECT:51.8515625
Allocated POST-concat BEFORE-COLLECT:54.5576171875
Allocated POST-concat  AFTER-COLLECT:51.8515625
Allocated POST-concat BEFORE-COLLECT:54.5576171875
Allocated POST-concat  AFTER-COLLECT:51.8515625

在程序initialization之后和調用CONTAINER.PopulateState() ，程序已經使用了 48.7 KB。

在對CONTAINER.PopulateState()的第一次調用中，增加了 3 KB 的內存，這似乎是持久的：該內存似乎沒有在程序執行中被釋放。 這可能是由於字節碼編譯、緩存或內部使用造成的。

但是CONTAINER.PopulateState()的以下執行通常使用 2.7 KB 的內存，並且每次都會釋放此內存。 程序行為似乎非常一致： CONTAINER.PopulateState()的執行不會使程序使用更多內存。 實際上，與程序的其余部分 (48 KB) 相比，函數CONTAINER.PopulateState() (2.7 KB) 使用的內存臨時可以忽略不計。

如果你想更好地控制正在發生的事情，你可以使用C語言來實現這部分，並提供一個Lua接口。

完整代碼：

CONTAINER =
{
      Ver = "0.3",
      --- integer lookup for the DateTime
      timeLUT = {[0]="00",[1]="01",[2]="02",[3]="03",[4]="04",[5]="05",[6]="06",[7]="07",[8]="08",[9]="09"},
      strCACHE = { [100] = ""},
      SubStrA  = "Unknown",
      SubAPrst = "ASjdasda",
}
     
    
for i = 10,99,1 do
  CONTAINER.timeLUT[i] = tostring(i)
end
        
DataBlob = {
  vAng = { x = 1.0, y = 2.0, z = 3.0},
  vPos = { x = 2131.0, y = 42.0, z = -433.0},
    
  Composite =
  {
        VARIANT1 = { isFirst = true, isMiddle = false, isLast = true },
        VARIANT2 = { isIgnored = true},
        VARIANT3 = { isAccurate = false },
        VARIANT4 = { bEnabled = false },
        VARIANT5 = { isLocked = false, ImpactV = 1.8 },
        VARIANT6 = { troCoWal = true },
        VARIANT7 = { isBroCal = false }
  } 
} 
    
Global = {
  isLocked = function(x)return false end,
  GetTimeStamp = function(x)return math.random() + math.random(1, 99) end,
  GetLocalTimeStamp = function(x)return math.random() + math.random(1, 99) end,
  GetTotalPTime = function(x)return math.random() + math.random(1, 99) end,
  GetDataBlob = function(x)return DataBlob end,
  GetName = function(x)return "AThing" end
}
    
function CONTAINER.PopulateState()
 
    local gcInit = 0
    local gcLast = 0
    
  -- Cachig globals
    
  local floor, mod, tostring = math.floor, math.mod, tostring
  local G = Global
  local intCache = CONTAINER.timeLUT
  local strBuilder = CONTAINER.strCACHE
    
  -- Fetching & caching data
  local locDB, Name = G.GetDataBlob(), G.GetName()
  local ts = G.GetTimeStamp()
  local lag = math.random() + math.random(1, 2)
    
  -- Local helpers
  local function sBool(bool)
    return bool and "1" or "0"
  end
    
  local t = 0
    
  function cAppend(cTbl, ...)
    for i=0, arg.n do
      cTbl[#cTbl+1] = arg[i]
      t = t +1
    end
  end

  function cClear(cTbl)
    for _=0, #cTbl do
      cTbl[#cTbl] = nil
    end
  end
        
  -- Populating table
  cClear(strBuilder)
        
  if locDB ~= nil then
    locDB = G.GetDataBlob()
    local PC = locDB.Composite
    local tp = G.GetTotalPTime()
    local d, h, m, s = floor(tp/86400), floor(mod(tp, 86400)/3600), floor(mod(tp,3600)/60), floor(mod(tp,60))
    
    cAppend(strBuilder,  "[", Name, "]:\n",
            "Ang :",      "(", tostring(locDB.vAng.x),",",tostring(locDB.vAng.y),",",tostring(locDB.vAng.z), ")\n",
            "Pos :",      "(", tostring(locDB.vPos.x),",",tostring(locDB.vPos.y),",",tostring(locDB.vPos.z), ")\n",
            "isLocked: ", sBool(G.isLocked()),  "\n")
        
    if (locDB.Composite["VARIANT1"] ~= nil) then
      cAppend(strBuilder, "isFirst / isLast: ", sBool(PC.VARIANT1.isFirst)," / ",sBool(PC.VARIANT1.isLast), "\n",   
              "isMiddle: ",         sBool(PC.VARIANT1.isMiddle), "\n")
    end
    
    if (locDB.Composite["VARIANT2"] ~= nil) then
      cAppend(strBuilder, "isIgnored: ",  sBool(PC.VARIANT2.isIgnored),  "\n")
    end
    
    if (locDB.Composite["VARIANT4"] ~= nil) then
      cAppend(strBuilder, "bEnabled: ",   sBool(PC.VARIANT4.bEnabled),   "\n")
    end
    
    if (locDB.Composite["VARIANT3"] ~= nil) then
      cAppend(strBuilder, "isAccurate: ", sBool(PC.VARIANT3.isAccurate), "\n")
    end
    
    if (locDB.Composite["VARIANT5"] ~= nil) then
      cAppend(strBuilder, "isLocked: ",   sBool(PC.VARIANT5.isLocked),   "\n",
              "ImpactV: ",    tostring(PC.VARIANT5.ImpactV), "\n")
    end
    
    if (locDB.Composite["VARIANT6"]) then
      cAppend(strBuilder, "troCoWal: ",   sBool(PC.VARIANT6.troCoWal),   "\n")
    end
    
    if (locDB.Composite["VARIANT7"]) then
          cAppend(strBuilder, "isBroCal: ",   sBool(PC.VARIANT7.isBroCal),   "\n")
    end
    
    cAppend(strBuilder, "Time taken: ",intCache[d],":",intCache[h],":",intCache[m],":",intCache[s], "\n",
    
                        "TS: ",        tostring(ts),                   "\n",    
                        "local TS: ",  tostring(G.GetLocalTimeStamp()),"\n",    
                        "Lag: ",       string.format("%.5f", lag) , " ms\n",    
                        "Heap: ",      tostring(gcLast),             "KB\n")
    
 
    
    cAppend(strBuilder, "Alloc: ",     tostring(gcLast-gcInit),"KB"," (v", CONTAINER.Ver, ")","\n",
    
                        "Extra: ",     CONTAINER.SubStrA, "_", CONTAINER.SubAPrst,             "\n")    
  end
end
    
 
print("Allocated PRE-concat:                " ..  tostring(collectgarbage("count")))

-- First time
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

-- One more try
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

-- One more try
CONTAINER.PopulateState()
print("Allocated POST-concat BEFORE-COLLECT:" ..  tostring(collectgarbage("count")))
collectgarbage("collect") 
print("Allocated POST-concat  AFTER-COLLECT:" ..  tostring(collectgarbage("count")))

Answer 2

您希望最小化字符串對象的中間分配次數，以降低 GC 壓力並減慢 GC 命中。 在這種情況下，我建議您將自己限制為 1 次使用要格式化的字符串對string.format調用：

• 字符串格式可以全局聲明，以便它被interned一次。
• string.format代碼可以在這里閱讀。 我們可以從這段代碼中看到，中間字符串轉換是在C堆棧上完成的，緩沖區為LUAL_BUFFERSIZE字節。 這個大小在luaconf.h聲明，可以根據你的需要定制。 這種方法對於您的用例應該是最有效的，因為您只需刪除所有中間步驟（表插入、table.concat 等）。

local MY_STRING_FORMAT = [[My Very Big String
param-string-1 %d
param-string-2 %x
param-string-3 %f
param-string-4 %d
param-string-5 %d
]]

string.format(MY_STRING_FORMAT,
              Param1,
              Param2,
              Param3,
              Param4,
              Param5,
              etc...)