Complex data structures with node-ffi

I recently got tasked with creating a node.js wrapper around one of our C libraries at work. Since I already created a java wrapper around the same C library, I knew what I needed to do and started looking into some options, which were to either create a node addon or using node-ffi. I was tired of writing stuff in C (a language I’ve barely used) since I made the java wrapper in JNI, I chose to try node-ffi.

Node-ffi was really easy to get started with, as long as your library functions used primitive datatypes. However, our C library uses a lot of structs, and that’s where things got complicated. Fortunately, there is the ref-struct node module, which makes it easy to define a struct. However, we had some string maps and string arrays that consisted of a struct holding a pointer and a length, something like this:

On the javascript side, it would be mapped like this:

Easy enough? Here’s where I got stuck. The C library would return a struct with a length of 7 (len set to 7).

You can probably get the issue, how do I get all of the elements? It took me quite a while, but here’s how I did:

This will create a javascript object of the StringMap, using the keys as object keys.

Let’s check the documentation for ref.get(buffer, offset, type). It accepts the buffer holding the data, an offset and the type. What we have to do is to iterate over the pointers, in steps of 4 bytes on 32-bit systems and 8 bytes on 64-bit systems. Using ref.sizeof.pointer we can get the correct size. This can also be used on a string array made the same way (a struct containing a pointer to a string (char *) and a length), just skip the key and push the return value of ref.get to an array).

But what about a struct containing an array of a different struct? Here it becomes complicated.

And on javascript:

The language struct takes 24 bytes to store on a 64-bit system (3 fields * 8 bytes). If you try to use the same code as we did for our string map:

You soon notice it will throw an exception that the passed in buffer needs to be at least 24 bytes. I assume it refers to language, so I tried to pass in a buffer with a size of 24 or more, or do a ref.alloc(language) since the documentation said it would create a buffer with the correct size and the correct type set. However, that would throw an error “could not determine a proper “type” from: [0,0,0,0,0,0,0,0,32,2,75,1,0,0,3,0,91,44,93,23,0,0,0,0]“. Well, not really helpful. I tried reading the documentation and trying out various things, but nothing worked. Then by sheer luck, when I was ready to give up, I managed to stumble upon one of the fields:

What the hell? So turns out that when calling ref.get with the type of ref.types.CString, we can get the values of the strings. They seem to come in the same order as the struct is defined, so at offset 0, we get the id field of the first struct, at offset 8 (element 0 + size of CString on a 64-bit os) we get the name field of element 0, at offset 16 we get the code field of element 0, at offset 24 we get the id field of element 1 and so on and so on. In this snippet, 3 is the number of fields in the struct, and is valid if each field is a string. If a field is something else than a string, you obviously need to make changes to the offset calculations and the size parameter (the last one).

This means you have to recreate your data structure and this may not be the correct way of doing it, but at least it works, and I hope it saves you from any headaches :D.