How to annotate struct fields with GObject annotations

I am having problems annotation a C struct with GObject annotations, so it can be accessed from Python. I am trying to write something like this:

/**
 * MyMatrixWrapper:
 * @matrix: (array fixed-size=16): (element-type gdouble): pointer to the matrix data
 */
typedef struct {
    double *matrix;
} MyMatrixWrapper

This seems to go through correctly, and in Python I correctly get a list 16 floating point values, but it seems like it is the actual pointer that has been casted to a float, instead of the array that the pointer is referencing:

[6.9253852512027e-310, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

Is there a better way to do this?