Integrating Lua with C: Part 5

To-be-closed Variables

As Lua is a garbage collected language, the Lua programmer does not have to care about memory management at all. Every value, once it is no longer used, will eventually be garbage collected and the memory it occupied will be be freed. This is also true for values that have been created by C code as userdata values.

While garbage collection is a great concept from the Lua programmers point of view, it is not optimal from the view of a C programmer who writes a Lua module in C. There is no control over when a value actually gets garbage collected. This can lead to situations where a lot of no longer used — and no longer accessible — memory is still being held by the process running the Lua interpreter.

As an example, imagine a PostgreSQL database interface being used in a Lua program that runs thousands of SQL queries per second. Each query returns a PostgreSQL result set that is essentially allocated memory that has to be explicitely freed by a call to the PQfree() C function. Whe could free the memory with an explicit call from the Lua program, but a Lua programmer might forget to call the free function, assuming everything is garbage collected.

On the other hand, if we free the memory at garbage collection time only, we have exactly the situation where potentially a lot of no longer used memory remains allocated.

Database result sets are only one example, we can think of a lot of other resources that are release to late or later than needed.

To address this problem, Lua 5.4 introduced to-be-closed variables, a mechanism to free memory or otherwise release resources at the very moment the variable goes out of scope.

To do so, two requirements must be met:

• The underlying userdata value must have a __close metamethod.
• The Lua program must annotate a variable as to-be-closed.

Consider the following function:

local function getName(db)
	local res = db:exec('select name from account')
	return res[1].name
end

The memory used by the variable res will only be freed at garbage collection time (whenever that will be.)

If, however, res is annotated as to-be-closed using the new keyword <close>, the underlying memory can be freed in the __close metamethod, which gets called as soon a the variable goes out of scope, i.e. when the getName() function returns:

local function getName(db)
	local res <close> = db:exec('select name from account') 
	return res[1].name
end

res is annoted as to-be-closed with the <close> keyword.

If you try to annotate a variable as to-be-closed when the underlying userdata has no __close metamethod, Lua will throw an error.

For the curious, this is the C function that is called in the Lua PostgreSQL interface to free memory either when __closeis called or when the garbage collector runs:

static int
res_clear(lua_State *L)
{
	PGresult **r;
	r = luaL_checkudata(L, 1, RES_METATABLE);
	if (*r) { 
		PQclear(*r);
		*r = NULL;
	}
	return 0;
}

Here we check the pointer *r to prevent double free of the memory as this function can be called more than one time.