[英]Passing a collection of JNA structures to native method
I am attempting to pass a collection of JNA structures to a native method but it's proving very fiddly:我试图将一组 JNA 结构传递给本机方法,但事实证明它非常繁琐:
Let's say we have a structure:假设我们有一个结构:
class MyStructure extends Structure {
// fields...
}
and a method in a JNA interface:和 JNA 接口中的一个方法:
void pass(MyStructure[] data);
which maps to the native method:它映射到本机方法:
void pass(const MYStructure* data);
Now the complication comes from the fact that the application is building a collection of these structures dynamically, ie we are NOT dealing with a static array but something like this:现在的复杂性来自这样一个事实,即应用程序正在动态地构建这些结构的集合,即我们不是在处理静态数组,而是在处理这样的事情:
class Builder {
private final Collection<MyStructure> list = new ArrayList<>();
// Add some data
public void add(MyStructure entry) {
list.add(entry);
}
// Pass the data to the native library
public void pass() {
// TODO
}
}
A naive implementation of the pass()
method could be: pass()
方法的简单实现可能是:
MyStructure[] array = list.toArray(MyStucture[]::new);
api.pass(array);
(where lib
is the JNA library interface). (其中
lib
是 JNA 库接口)。
Of course this doesn't work because the array is not a contiguous block of memory - fair enough.当然这不起作用,因为数组不是连续的内存块 - 足够公平。
One solution is to allocate a JNA array from a structure instance and populate it field-by-field:一种解决方案是从结构实例分配一个 JNA 数组并逐个字段填充它:
MYStructure[] array = (MyStructure[]) new MyStructure().toArray(size);
for(int n = 0; n < array.length; ++n) {
array[n].field = list.get(n).field;
// other fields...
}
This guarantees the array consist of contiguous memory.这保证了数组由连续的内存组成。 But we have had to implement a field-by-field copy of the data (which we've already populated in the list) - this is OK for a simple structure, but some of the data I am dealing with has dozens of fields, structures that point to further nested arrays, etc. Basically this approach is just not viable.
但是我们必须实现数据的逐个字段副本(我们已经在列表中填充了它)——这对于一个简单的结构来说是可以的,但是我正在处理的一些数据有几十个字段,指向进一步嵌套数组等的结构。基本上这种方法是不可行的。
Another alternative is to convert the collection of data to a simple JNA pointer, something along these lines:另一种选择是将数据集合转换为一个简单的 JNA 指针,大致如下:
MyStructure[] array = list.toArray(MyStructure[]::new);
int size = array[0].size();
Memory mem = new Memory(array.length * size);
for(int n = 0; n < array.length; ++n) {
if(array[n] != null) {
array[n].write();
byte[] bytes = array[n].getPointer().getByteArray(0, size);
mem.write(n * size, bytes, 0, bytes.length);
}
}
This solution is generic so we can apply it to other structure as well.此解决方案是通用的,因此我们也可以将其应用于其他结构。 But we have to change the method signatures to be
Pointer
instead of MyStructure[]
which makes the code more obtuse, less self-documenting and harder to test.但是我们必须将方法签名更改为
Pointer
而不是MyStructure[]
,这使得代码更钝,更少的自我记录和更难测试。 Also we could be using a third-party library where this might not even be an option.此外,我们可能会使用第三方库,而这甚至可能不是一种选择。
(Note I asked a similar question a while ago here but didn't get a satisfactory answer, thought I'd try again and I'll delete the old one / answer both). (注意我刚才在这里问了一个类似的问题,但没有得到满意的答案,我想我会再试一次,我会删除旧的/回答两个)。
Basically I was expecting/hoping to have something like this:基本上我期待/希望有这样的事情:
MyStructure[] array = MyStructure.magicContiguousMemoryBlock(list.toArray());
similar to how the JNA helper class provides StringArray
for an array-of-string:类似于 JNA 助手类为字符串数组提供
StringArray
的方式:
StringArray array = new StringArray(new String[]{...});
But no such 'magic' exists as far as I can tell.但据我所知,不存在这样的“魔法”。 Is there another, simpler and more 'JNA' way of doing it?
有没有另一种更简单、更“JNA”的方式来做到这一点? It seems really dumb (and probably incorrect) to have to allocate a byte-by-byte copy of the data that we essentially already have!
必须为我们基本上已经拥有的数据分配一个逐字节的副本似乎真的很愚蠢(并且可能不正确)!
Do I have any other options?我还有其他选择吗? Any pointers (pun intended) gratefully accepted.
感激地接受任何指示(双关语)。
If you able to create a continues block of memory, why don't you simply de-serialize your list into it.如果您能够创建一个连续的内存块,为什么不简单地将您的列表反序列化为它。
Ie something like:即类似:
MyStructure[] array = list.get(0).toArray(list.size());
list.toArray(array);
pass(array);
In any case you'd better not to store Structure
in your List or any another collection.在任何情况下,您最好不要将
Structure
存储在您的 List 或任何其他集合中。 It is better idea to hold a POJO inside, and then remap it to array of structures directly using a bean mapping library or manually.最好在内部保存一个 POJO,然后直接使用bean 映射库或手动将其重新映射到结构数组。
With MapStruct bean mapping library it may looks like:使用MapStruct bean 映射库,它可能看起来像:
@Mapper
public interface FooStructMapper {
FooStructMapper INSTANCE = Mappers.getMapper( FooStructMapper.class );
void update(FooBean src, @MappingTarget MyStruct dst);
}
MyStrucure[] block = new MyStructure().toArray(list.size());
for(int i=0; i < block.length; i++) {
FooStructMapper.INSTANCE.update(list.get(i), block[i]);
}
What the point - Structure constructor allocates memory block using Memory, it is really slow operation.重点是什么 - 结构构造函数使用内存分配内存块,它的操作确实很慢。 As well as memory allocated outside of java heap space.
以及在 java 堆空间之外分配的内存。 It is always better to avoid this allocate whenever you can.
尽可能避免这种分配总是更好。
As the author of the previous answer, I realize a lot of the confusion was approaching it one way before realizing a better solution that we discussed primarily in comments to your answer.作为上一个答案的作者,我意识到在实现我们主要在对您的答案的评论中讨论的更好的解决方案之前,很多混乱都是以一种方式接近它的。 I will try to answer this additional clarification with an actual demonstration of my suggestion on that answer which I think is the best approach.
我将尝试通过实际演示我对该答案的建议来回答这个额外的澄清,我认为这是最好的方法。 Simply, if you have a non-contiguous structure and need a contiguous structure, you must either bring the contiguous memory to the structure, or copy the structure to the contiguous memory.
简单来说,如果你有一个不连续的结构并且需要一个连续的结构,你必须要么把连续的内存带到结构中,要么把结构复制到连续的内存中。 I'll outline both approaches below.
我将在下面概述这两种方法。
Is there another, simpler and more 'JNA' way of doing it?
有没有另一种更简单、更“JNA”的方式来做到这一点? It seems really dumb (and probably incorrect) to have to allocate a byte-by-byte copy of the data that we essentially already have!
必须为我们基本上已经拥有的数据分配一个逐字节的副本似乎真的很愚蠢(并且可能不正确)!
I did mention in my answer on the other question that you could use useMemory()
in this situation.我在另一个问题的回答中确实提到在这种情况下您可以使用
useMemory()
。 It is a protected
method but if you are already extending a Structure
you have access to that method from the subclass (your structure), in much the same way (and for precisely the same purpose) as you would extend the Pointer
constructor of a subclass.这是一个
protected
方法,但如果您已经扩展了一个Structure
您可以从子类(您的结构)访问该方法,其方式与扩展子类的Pointer
构造函数大致相同(并且目的完全相同) .
You could therefore take an existing structure in your collection and change its native backing memory to be the contiguous memory.因此,您可以采用集合中的现有结构并将其本机后备内存更改为连续内存。 Here is a working example:
这是一个工作示例:
public class Test {
@FieldOrder({ "a", "b" })
public static class Foo extends Structure {
public int a;
public int b;
// You can either override or create a separate helper method
@Override
public void useMemory(Pointer m) {
super.useMemory(m);
}
}
public static void main(String[] args) {
List<Foo> list = new ArrayList<>();
for (int i = 1; i < 6; i += 2) {
Foo x = new Foo();
x.a = i;
x.b = i + 1;
list.add(x);
}
Foo[] array = (Foo[]) list.get(0).toArray(list.size());
// Index 0 copied on toArray()
System.out.println(array[0].toString());
// but we still need to change backing memory for it to the copy
list.get(0).useMemory(array[0].getPointer());
// iterate to change backing and write the rest
for (int i = 1; i < array.length; i++) {
list.get(i).useMemory(array[i].getPointer());
list.get(i).write();
// Since sending the structure array as an argument will auto-write,
// it's necessary to sync it here.
array[1].read();
}
// At this point you could send the contiguous structure array to native.
// Both list.get(n) and array[n] point to the same memory, for example:
System.out.println(list.get(1).toString());
System.out.println(array[1].toString());
}
Output (note the contiguous allocation).输出(注意连续分配)。 The second two outputs are the same, from either the list or the array.
后两个输出是相同的,来自列表或数组。
Test$Foo(allocated@0x7fb687f0d550 (8 bytes) (shared from auto-allocated@0x7fb687f0d550 (24 bytes))) {
int a@0x0=0x0001
int b@0x4=0x0002
}
Test$Foo(allocated@0x7fb687f0d558 (8 bytes) (shared from allocated@0x7fb687f0d558 (8 bytes) (shared from allocated@0x7fb687f0d558 (8 bytes) (shared from allocated@0x7fb687f0d550 (8 bytes) (shared from auto-allocated@0x7fb687f0d550 (24 bytes)))))) {
int a@0x0=0x0003
int b@0x4=0x0004
}
Test$Foo(allocated@0x7fb687f0d558 (8 bytes) (shared from allocated@0x7fb687f0d558 (8 bytes) (shared from allocated@0x7fb687f0d550 (8 bytes) (shared from auto-allocated@0x7fb687f0d550 (24 bytes))))) {
int a@0x0=0x0003
int b@0x4=0x0004
}
If you don't want to put useMemory
in every one of your structure definitions you can still put it in an intermediate class that extends Structure
and then extend that intermediate class instead of Structure
.如果您不想将
useMemory
放在每个结构定义中,您仍然可以将它放在扩展Structure
的中间类中,然后扩展该中间类而不是Structure
。
If you don't want to override useMemory()
in your structure definitions (or a superclass of them), you can still do it "simply" in code with a little bit of inefficiency by copying over the memory.如果您不想在您的结构定义(或它们的超类
useMemory()
中覆盖useMemory()
,您仍然可以通过复制内存来“简单地”在代码中“简单”地完成它,但效率低下。
In order to "get" that memory to write it elsewhere, you have to either read it from the Java-side memory (via reflection, which is what JNA does to convert the structure to the native memory block), or read it from Native-side memory (which requires writing it there, even if all you want to do is read it).为了“获取”该内存以将其写入其他地方,您必须从 Java 端内存中读取它(通过反射,这是 JNA 将结构转换为本机内存块所做的工作),或者从 Native 中读取它-side 内存(需要将它写入那里,即使您想做的只是读取它)。 Under-the-hood, JNA is writing the native bytes field-by-field, all hidden under a simple
write()
call in the API.在底层,JNA 正在逐个字段写入本机字节,所有这些都隐藏在 API 中的简单
write()
调用下。
Your "Rubbish Solution #2" seems close to what's desired in this case.在这种情况下,您的“垃圾解决方案 #2”似乎与所需的很接近。 Here are the constraints that we have to deal with, with whatever solution:
以下是我们必须处理的约束,无论采用何种解决方案:
Structure
, the native memory is not contiguous (unless you pre-allocate contiguous memory yourself, and use that memory in a controlled manner, or override useMemory()
as demonstrated above), and the size is variable.Structure
的现有列表或数组中,本机内存不是连续的(除非您自己预先分配连续的内存,并以受控方式使用该内存,或者如上所述覆盖useMemory()
),并且大小是可变的。 Here are the "JNA ways" of dealing with structures and memory:以下是处理结构和内存的“JNA 方式”:
Structure.getPointer()
with a size of (at least) Structure.size()
.Structure.getPointer()
访问,其大小(至少)为Structure.size()
。Structure.getByteArray()
.Structure.getByteArray()
批量读取结构本机内存。new Structure(Pointer p)
constructor.new Structure(Pointer p)
构造函数从指向本机内存的new Structure(Pointer p)
构造结构。Structure.toArray()
method creates an array of structures backed by a large, contiguous block of native memory. Structure.toArray()
方法创建一个由大的、连续的本机内存块支持的结构数组。 I think your solution #2 is a rather efficient way of doing it, but your question indicates you'd like more type safety, or at least self-documenting code, in which case I'd point out a more "JNA way" of modifying #2 with two steps:我认为您的解决方案 #2 是一种相当有效的方法,但是您的问题表明您想要更多的类型安全,或者至少是自记录代码,在这种情况下,我会指出一种更“JNA 方式”的用两个步骤修改 #2:
Memory(array.length * size)
native allocation with the Structure.toArray()
allocation from your solution #1.Structure.toArray()
分配替换新的Memory(array.length * size)
本机分配。
length * size
block of contiguous native memory and a pointer to it ( array[0].getPointer()
).length * size
的连续本机内存块和一个指向它的指针( array[0].getPointer()
)。mem.write(n * size, ... )
with array[n].getPointer().write(0, ... )
.mem.write(n * size, ... )
替换为array[n].getPointer().write(0, ... )
。getByteArray()
and immediately write()
that byte array seem clear enough to me.getByteArray()
并立即write()
字节数组对我来说似乎足够清楚。
write(0, getByteArray(0, size), 0, size)
, although one might argue if that's more or less clear.write(0, getByteArray(0, size), 0, size)
一行... write(0, getByteArray(0, size), 0, size)
,尽管人们可能会争论这是否或多或少是清楚的。 So, adapting your method #2, I'd suggest:所以,调整你的方法#2,我建议:
// Make your collection an array as you do, but you could just keep it in the list
// using `size()` and `list.get(n)` rather than `length` and `array[n]`.
MyStructure[] array = list.toArray(MyStructure[]::new);
// Allocate a contiguous block of memory of the needed size
// This actually writes the native memory for index 0,
// so you can start the below iteration from 1
MyStructure[] structureArray = (MyStructure[]) array[0].toArray(array.length);
// Iterate the contiguous memory and copy over bytes from the array/list
int size = array[0].size();
for(int n = 1; n < array.length; ++n) {
if(array[n] != null) {
// sync local structure to native (using reflection on fields)
array[n].write();
// read bytes from the non-contiguous native memory
byte[] bytes = array[n].getPointer().getByteArray(0, size);
// write bytes into the contiguous native memory
structureArray[n].getPointer().write(0, bytes, 0, bytes.length);
// sync native to local (using reflection on fields)
structureArray[n].read();
}
}
From a "clean code" standpoint I think this rather effectively accomplishes your goal.从“干净的代码”的角度来看,我认为这相当有效地实现了您的目标。 The one "ugly" part of the above method is that JNA doesn't provide an easy way to copy fields between Structures without writing them to native memory in the process.
上述方法的一个“丑陋”部分是 JNA 没有提供一种简单的方法来在 Structures 之间复制字段而不在过程中将它们写入本机内存。 Unfortunately that's the "JNA way" of "serializing" and "deserializing" objects, and it's not designed with any "magic" for your use case.
不幸的是,这是“序列化”和“反序列化”对象的“JNA 方式”,并且它没有为您的用例设计任何“魔法”。 Strings include built-in methods to convert to bytes, making such "magic" methods easier.
字符串包括转换为字节的内置方法,使这种“神奇”的方法更容易。
It is also possible to avoid writing the structure to native memory just to read it back again if you do the field-by-field copy as you implied in your Method #1.如果您按照方法 #1 中暗示的那样进行逐字段复制,也可以避免将结构写入本机内存只是为了再次读取它。 However, you could use JNA's field accessors to make it a lot easier to access the reflection under the hood.
但是,您可以使用 JNA 的字段访问器来更轻松地访问引擎盖下的反射。 The field methods are
protected
so you'd have to extend Structure
to do this -- which if you're doing that, the useMemory()
approach is probably better!字段方法是
protected
因此您必须扩展Structure
才能执行此操作——如果您这样做, useMemory()
方法可能更好! But you could then pull this iteration out of write()
:但是你可以把这个迭代从
write()
拉出来:
for (StructField sf : fields().values()) {
// do stuff with sf
}
My initial thought would be to iterate over the non-contiguous Structure
fields using the above loop, storing a Field.copy()
in a HashMap
with sf.name
as the key.我最初的想法是使用上述循环遍历非连续的
Structure
字段,将Field.copy()
存储在sf.name
作为键的HashMap
。 Then, perform that same iteration on the other (contiguous) Structure
object's fields, reading from the HashMap
and setting their values.然后,对另一个(连续的)
Structure
对象的字段执行相同的迭代,从HashMap
读取并设置它们的值。
The solutions offered by Daniel Widdis will solve this 'problem' if one really needs to perform a byte-by-byte copy of a JNA structure.如果确实需要逐字节复制 JNA 结构, Daniel Widdis提供的解决方案将解决这个“问题”。
However I have come round to the way of thinking expressed by some of the other posters - JNA structures are intended purely for marshalling to/from the native layer and should not really be used as 'data'.然而,我已经转向其他一些海报表达的思维方式 - JNA 结构纯粹用于编组到/来自本机层,不应真正用作“数据”。 We should be defining domain POJOs and transforming those to JNA structures as required - a bit more work but deal with I guess.
我们应该定义域 POJO 并根据需要将它们转换为 JNA 结构 - 需要做更多的工作,但我想可以处理。
EDIT: Here is the solution that I eventually implemented using a custom stream collector:编辑:这是我最终使用自定义流收集器实现的解决方案:
public class StructureCollector <T, R extends Structure> implements Collector<T, List<T>, R[]> {
/**
* Helper - Converts the given collection to a contiguous array referenced by the <b>first</b> element.
* @param <T> Data type
* @param <R> Resultant JNA structure type
* @param data Data
* @param identity Identity constructor
* @param populate Population function
* @return <b>First</b> element of the array
*/
public static <T, R extends Structure> R toArray(Collection<T> data, Supplier<R> identity, BiConsumer<T, R> populate) {
final R[] array = data.stream().collect(new StructureCollector<>(identity, populate));
if(array == null) {
return null;
}
else {
return array[0];
}
}
private final Supplier<R> identity;
private final BiConsumer<T, R> populate;
private final Set<Characteristics> chars;
/**
* Constructor.
* @param identity Identity structure
* @param populate Population function
* @param chars Stream characteristics
*/
public StructureCollector(Supplier<R> identity, BiConsumer<T, R> populate, Characteristics... chars) {
this.identity = notNull(identity);
this.populate = notNull(populate);
this.chars = Set.copyOf(Arrays.asList(chars));
}
@Override
public Supplier<List<T>> supplier() {
return ArrayList::new;
}
@Override
public BiConsumer<List<T>, T> accumulator() {
return List::add;
}
@Override
public BinaryOperator<List<T>> combiner() {
return (left, right) -> {
left.addAll(right);
return left;
};
}
@Override
public Function<List<T>, R[]> finisher() {
return this::finish;
}
@SuppressWarnings("unchecked")
private R[] finish(List<T> list) {
// Check for empty data
if(list.isEmpty()) {
return null;
}
// Allocate contiguous array
final R[] array = (R[]) identity.get().toArray(list.size());
// Populate array
final Iterator<T> itr = list.iterator();
for(final R element : array) {
populate.accept(itr.next(), element);
}
assert !itr.hasNext();
return array;
}
@Override
public Set<Characteristics> characteristics() {
return chars;
}
}
This nicely wraps up the code that allocates and populates a contiguous array, example usage:这很好地包装了分配和填充连续数组的代码,示例用法:
class SomeDomainObject {
private void populate(SomeStructure struct) {
...
}
}
class SomeStructure extends Structure {
...
}
Collection<SomeDomainObject> collection = ...
SomeStructure[] array = collection
.stream()
.collect(new StructureCollector<>(SomeStructure::new, SomeStructure::populate));
Hopefully this might help anyone that's doing something similar.希望这可以帮助任何正在做类似事情的人。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.