Circular Reference in HashMap

[bbaanngg 0]

m = createHashMap;
map = createHashMap;
map set ["a", m];
map set ["b", m];

If you run this code in the debug console and then attempt to save the mission the following message will be logged to the rpt file:

Circular reference found in HashMap, this will cause a memory leak!
➥ Context: Hash Key: "b" <- Serialize Game Variable: map

However if you run the following code:

a = [];
map = createHashMap;
map set ["a", a];
map set ["b", a];

No such error is logged.

 

Can anyone explain this behavior? Specifically, why can you store two references to the same array but not to the same hashmap? Thanks.

[mrcurry 497]

23 hours ago, bbaanngg said:

Can anyone explain this behavior?

 

Honestly seems like a bug. Good find if so, I wonder does it only happen when 'm' is empty? Or if it's not a missionNamespace variable?

 

Last I checked loaded HashMaps doesn't retain group or object references either, unlike arrays. My guess is the serializer doesn't know how to handle them properly.

 

You should post it to the Feedback Tracker  but I wouldn't hold my breath for a fix expect a fix. It's quite a niche issue and you can work around it by converting HashMaps to arrays on save, then use the Loaded eventhandler and createHashMapFromArray to rebuild.

[soldierXXXX 108]

Edit:

Please ignore this comment. I was wrong.Values i was referring to do not have not be hashable.

 

To me it doesn't seem like a bug, but rather like unsupported type of the value, because value that you give to the key must be hashable to be properly stored. You can find here all the supported types you can use as value with hashmap. Further reading how hashmap works is here. What might be happening is that hashmaps somehow create strong reference on each other and cannot be released from the memory resulting into memory leak, but I'm not a programmer so i don't know for sure, also we don't have direct access to the engine so we cannot check. I might be wrong on that though. Question is what is the use case of having hashmap inside hashmap? Why cannot be either array of hashmaps or hashmap with key and array value? 

Edited April 15 by soldierXXXX

[mrcurry 497]

29 minutes ago, soldierXXXX said:

because value that you give to the key must be hashable to be properly stored

The key in this case is a string, "a" or "b" are very hashable.

The stored value is the reference to the first HashMap.

 

29 minutes ago, soldierXXXX said:

Question is what is the use case of having hashmap inside hashmap? Why cannot be either array of hashmaps or hashmap with key and array value? 

Nesting HashMaps like this is very useful for quick lookups of complex nature. Example: You have some thing that doesn't support setVariable, you can use nested hashmaps to replicate similar functionality where the lower level contains "variables" and their values while the upper level acts as a registry for each "thing".

The lower level keys can be shared across multiple things which means easy to use and value lookup is fast even with massive amounts of objects with a lot of variables.

 

And to clarify: This works fine during runtime. This is only a problem during save/load of a mission.

 

A bug report is also a way to get verification if it is or isn't a bug. 🙂

 

[soldierXXXX 108]

Ah alright cool. That makes sense. I thought that value should also be hashable, but looking at the syntax it could be anything. So in that case yes that is indeed a bug. Thanks for clarifying 🙂.

[bbaanngg 0]

7 hours ago, mrcurry said:

Honestly seems like a bug.

I agree.

 

25 minutes ago, soldierXXXX said:

To me it doesn't seem like a bug, but rather like unsupported type of the value, because value that you give to the key must be hashable to be properly stored.

Only the keys of a hashmap need to be hashable.

From https://community.bistudio.com/wiki/HashMap:

Quote

When a key-value pair is inserted into a HashMap, a hash function is applied to the key to determine the position at which the key-value pair is going to be stored. Then, when we want to retrieve the value associated with a specific key, the same hash function (Arma 3 uses FNV-1a 64-bit) is applied to that key and the resulting position tells the HashMap exactly where to find the key-value pair that we are looking for.

 

7 hours ago, mrcurry said:

I wonder does it only happen when 'm' is empty?

I generalized the problem for the sake of this post. In the actual implementation the maps are not empty, so that doesn't matter.

 

25 minutes ago, soldierXXXX said:

Question is what is the use case of having hashmap inside hashmap? Why cannot be either array of hashmaps or hashmap with key and array value? 

My original problem consisted of storing an array of maps and a subset of that array under separate keys in the same parent hashmap. The parent map is a "point cloud" object created using createHashMapObject. The two arrays are populated with "point" object hashmaps. The first array describes all points of the point cloud, and the second array is the subset of the first array that creates the convex hull of the point cloud. See:

p1 = createHashMapObject [...];
p2 = createHashMapObject [...];
...

PointCloud = createHashMapObject [...];
PointCloud set ["points", [p1, p2, ...]];
PointCloud set ["convex_hull", [p1, ...]];

 

7 hours ago, mrcurry said:

It's quite a niche issue and you can work around it by converting HashMaps to arrays on save, then use the Loaded eventhandler and createHashMapFromArray to rebuild.

The array of "points" will never change in my use case, so I can just store the indexes of the desired points in the "convex_hull" array to avoid the "circular reference".

However, your solution sounded good for a scenario where the first array is subject to change, so I did some testing.

I figured the problem with converting to an array would be keeping both values referencing the same map, and that seems to be the case: (note that specifying non-split arrays with toArray is required for createHashMapFromArray to work when using nested hashmaps.)

_m = createHashMapFromArray [["a", 1]];   
p = createHashMapFromArray [["m1", _m], ["m2", _m]];

[missionNamespace, "OnSaveGame", {
	p = p toArray false;
}] call BIS_fnc_addScriptedEventHandler;

addMissionEventHandler ["Loaded", {
	params ["_saveType"];
	p = createHashMapFromArray p; 
}];

Placing the above code in the init.sqf and running the mission looks promising at first. The save event handler properly avoids the circular reference error, and loading the save also restores the hashmap, but unfortunately, if you run the following code after loading the save:

p get "m1" set ["a", 2];  
p get "m2" get "a";

It does not return 2. Which shows that "m1" and "m2" no longer reference the same map.

 

However, if you run the following code and then save the game:

a = call {
 _m = createHashMapFromArray [["a", 1]];   
 _p = createHashMapFromArray [["m1", _m], ["m2", _m]];
 _p toArray false; 
};

Then test the reference using the following code:

p = createHashMapFromArray a; 
p get "m1" set ["a", 2];  
p get "m2" get "a";

If you load the save and then test the reference, "m1" and "m2" no longer reference the same map. However, if you save the game and test the reference without loading the save, the reference is retained.

 

7 hours ago, mrcurry said:

You should post it to the Feedback Tracker

Will do, thanks for your help.

[mrcurry 497]

14 hours ago, bbaanngg said:

Placing the above code in the init.sqf and running the mission looks promising at first. The save event handler properly avoids the circular reference error, and loading the save also restores the hashmap, but unfortunately, if you run the following code after loading the save:

p get "m1" set ["a", 2];  
p get "m2" get "a";

It does not return 2. Which shows that "m1" and "m2" no longer reference the same map.

 

Yeah that fits the bill with the other issue I mentioned but as long as the object structure is static it should be trivial to deconstruct/reconstruct the references manually.

 

P.s. out of curiosity what possessed you to do pointclouds in sqf? 😛