Garbage Collector and FFI

[RNavega]

Come on, that's not the entire code.
You have a 'ffi' name in there, so it's missing at least the require("ffi") call. Also, the indentation suggests this is within some block scope which does impact on the lifetime of objects.

The only time I had a garbage collection issue like that (quite a mysterious bug, as it would fail at a different time on each run of the program), it was when I was expecting the FFI pointer to keep the buffer object alive. It didn't.
After I stored both the FFI pointer and all the objects related to that pointer (the buffer, the data etc) in the same table, and had the table survive for some time, then it worked fine.

[UnixRoot]

Come on, that's not the entire code.

That's true, but i simply can't post my entire game.

After I stored both the FFI pointer and all the objects related to that pointer (the buffer, the data etc) in the same table, and had the table survive for some time, then it worked fine.

And how do you keep objects alive without packing everything into tables? I mean, I fill some of those buffers only one time while generating the map and then render them every frame. Other buffers are worked with constantly, they get new data every frame, and they still get garbage collected instantly. They don't get collected after some time, I start the program, boom, gone.

[RNavega]

I mean, I fill some of those buffers only one time while generating the map and then render them every frame. Other buffers are worked with constantly, they get new data every frame, and they still get garbage collected instantly.

In your position, I'd check these things:

• That all FFI objects are kept in variables (or tables inside those variables) that exist either in the global scope (the _G table) or the top-level local scope (declaring "local (...)" out in the open, outside any functions).

• That any buffers referenced by pointers will survive for at least as long as those pointers are used. The same for the LÖVE Data objects from which the pointers are created.

• Double-checking, with a few IFs, that my pointer addressing doesn't read outside the buffer boundaries, which would crash the program in a similar way.

[UnixRoot]

• That all FFI objects are kept in variables (or tables inside those variables) that exist either in the global scope (the _G table) or the top-level local scope (declaring "local (...)" out in the open, outside any functions).

But I already do this. They're stored in top-level local scope variables.

• That any buffers referenced by pointers will survive for at least as long as those pointers are used. The same for the LÖVE Data objects from which the pointers are created.

Yeah, and that's my biggest question. How do I keep them alive, they get garbage collected as soon as I start the program. Changing pixels on the image data objects, via pointers, doesn't keep them alive. Refreshing the pixels of the final image with imagedata doesn't keep them alive either.


Of course I check to not read any memory out of bounds, but it still reads garbage, because everything is being garbage collected instantly. If I turn off the garbage collector, all my problems are gone completely.

[pgimeno]

Come on, that's not the entire code.

That's true, but i simply can't post my entire game.

No one is asking you to do that. What we're asking for is a self-contained test program that can reproduce the problem.

I've modified your code by adding this at the beginning of your code:

Code: Select all

local ffi = require('ffi')

and this at the end:

Code: Select all

function love.update()
  collectgarbage('collect')
end

and then adding a sample public-domain image I had lying around and naming it 'tex.png'. The result is attached.

That example doesn't crash for me in 11.4. Does it for you?

Also, what version of Löve are you using, and what OS are you running it on? I'm using 11.4 on Linux.

Note also this crash in bitser, which I diagnosed (the "ghost" user in that thread is me): https://github.com/gvx/bitser/issues/9

The cause was that the lifetime of a pointer variable finished after a tail call was executed. See the patch that fixes it, and the full discussion.

[RNavega]

Yeah, and that's my biggest question. How do I keep them alive, they get garbage collected as soon as I start the program.

Since you already seem to be keeping references to the things being used (at least the obvious ones), I'd think that the problem might be happening from something not obvious.

For example, in your code:

Code: Select all

local fb = ffi.cast("uint16_t*", framebuffer:getFFIPointer())

Making sure to store a reference to the result of that getFFIPointer() instead of making it a throwaway, besides the object that it's coming from, and the FFI pointer.
That is, keeping a reference to absolutely everything used, since the obvious references aren't being enough.

[UnixRoot]

Yeah, and that's my biggest question. How do I keep them alive, they get garbage collected as soon as I start the program.

Since you already seem to be keeping references to the things being used (at least the obvious ones), I'd think that the problem might be happening from something not obvious.

For example, in your code:

Code: Select all

local fb = ffi.cast("uint16_t*", framebuffer:getFFIPointer())

Making sure to store a reference to the result of that getFFIPointer() instead of making it a throwaway, besides the object that it's coming from, and the FFI pointer.
That is, keeping a reference to absolutely everything used, since the obvious references aren't being enough.

Of course, I tried that too, storing the pointer in an extra variable. It still gets garbage collected. The only thing that prevents it from being garbage collected instantly after program start, is to pack everything into LUA tables or switching off the garbage collector.

[pgimeno]

This is a modified version of your code that does crash for me (for good reasons), using the image from the .love I posted above:

Code: Select all

local ffi = require('ffi')
local texBuffer
local terrainBuffer
local backgroundBuffer
do
    local imagedata=love.image.newImageData(640,360, "rgba8")
    local image = love.graphics.newImage(imagedata)
    terrainBuffer  = ffi.cast('uint32_t*', imagedata:getFFIPointer())

    local imagedata2 = love.image.newImageData(640,360, "rgba8")
    local image2 = love.graphics.newImage(imagedata2)
    backgroundBuffer = ffi.cast('uint32_t*', imagedata2:getFFIPointer())

    local texdata = love.image.newImageData("tex.png")
    texBuffer = ffi.cast('uint32_t*', texdata:getFFIPointer())
end

function love.update()
  collectgarbage('collect')
  texBuffer[0] = 1
  terrainBuffer[0] = 2
  backgroundBuffer[0] = 3
end

[Imagic]

Hi,

Reading the thread it is likely that you are affected by LuaJIT issue#1167.

Unfortunately the interaction between the Lua and C semantics through the FFI is not trivial. You can try the proposed workarounds to force the JIT compiler analysis (or in general the VM) to see the object as used until LuaJIT offers a way to specify that with the FFI semantics.