Subscribe to this thread
Home - General / All posts - M9 Unknown error. Unexpected operation in large overlays
danb


1,970 post(s)
#16-Nov-22 22:56

I am wondering if anyone has noticed an increasing incidence of the following error message with recent builds of Manifold 9?

I have been recently undertaking a number of admittedly large intersection operations and many of them have started to fail recently. I sent one into tech to see if there was anything they could spot but I don’t know the outcome. The particular intersection task I sent to tech used to complete without issue but seemingly no longer thought the data will have changed little.

The machines I have tried it on have plenty of resource and cache size is set to auto. There is over 1TB free on the C:\ SSD of the first machine and similar for the second which is also an SSD.

I find myself irked to have to be forced to complete these intersection tasks using ArcGIS Pro or even ArcMap 32bit on the same machine!

Additional: I exported the two current layers to geopackage from M9 and linked them into QGIS 3.20.3. Although it drew both layers fine, It reports a ton of geometry errors for both layers which are skipped during the intersection. It does however complete fine for the valid polygons, though the intersection data in purple should be for the full extent defined by the returned polygons.

Attachments:
Intersection in QGIS.png
Machine 1.png
Machine 2.png


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri

7,115 post(s)
#17-Nov-22 05:52

When you contacted tech support, did you report a bug or did you use a support token? I assume it was the former or you would have known the outcome, true? If it was the former, you don't say what the discussion, if any, was with tech support. The usual case in such instances is that it is not a bug but a machine issue. Classic examples are issues with data on network shares that might disconnect when in use, or insufficient memory.

For example, you don't say whether the "admittedly large" operations might exceed 1 TB in working space. Keep in mind the guidance in the "Memory" section of the Performance Tips topic:

  • Temp files can expand to three or more times the size of the actual project.

I once saw an increasing incidence of that error with 9, but it was only on one machine. Doing the same work on other machines didn't throw that error. So I assumed the problem was with storage. I had plenty of free space on SSD and HD, way more than ten times the size of the data I was working with, which indicated that lack of working space was not the issue.

I did recall that when I first configured that machine I had issues getting it to boot and then to load Windows onto the M4 SSD. Something seemed weird depending on which GPU card I inserted. I had ended up installing a less extreme GPU. Over time, that machine also threw errors in plain Windows operation, like copying very large files through the network to load up the HD.

I guessed that the problem might be either a flaky SSD or a problem with the hard disk. Anybody who has gone down that debugging path knows they have a long road ahead of them, since just running sensible diagnostics on very large disks takes a long time, and then swapping them out and reloading the new disk also takes a very long time.

To make a long story short, I did all that, changing SSD and HD and the machine seemed fine. And then when the new disk was loaded up again after a while errors started again, first being noticed in non-Manifold work (a fairly certain indication the problem is not an error in Manifold).

After a few months using other machines, I decided to once and for all figure out what the problem was, being ready to systematically swap out all components with "known good" components right up to the CPU and motherboard if need be, stripping the machine down to the simplest possible configuration and testing the heck out of everything.

The problem turned out to be one of the memory sticks, which occasionally threw errors at certain clock/voltage configurations. It's very high-quality memory but still, it was not totally reliable in configurations that it was supposed to run. Running it at reduced speeds worked fine.

I rebuilt the machine without the weak memory stick using all of the original components which I had replaced, thinking they might have been the problem. All of the problems noted are gone. I can use the more extreme GPU and there are zero issues with booting. I don't see any errors in Windows operations and the "unexpected error" in Manifold operations in exactly those same projects and operations has not recurred. I've been using that machine for over nine months now and it's run flawlessly.

Just saying, the error you report could be a bug in Manifold, or it could be something else. If you are working with tech support, they always keep in mind the possibility of a bug, even though those are rare. But the only way to tell for sure if it is a bug and not a machine issue like I reported above in my case, either a hardware problem, network issue, or an operational (lack of required storage) issue, can require a fairly lengthy exploration of all the details involved and can require very time-consuming trials using different configurations.

One thing you could do that might be easier would be to try exactly the same intersection operations that throw the above error, but do that on a greatly reduced data set on purely local storage (you didn't say explicitly that all work was done locally, so it's not safe to assume that it was....). That will have the advantage of likely excluding storage space as an issue. If an algorithm is flawed (a bug in Manifold), it's likely to be flawed with smaller data the same as with bigger data. Also, working with smaller data goes quicker so it's easier to try different variations to zero in on possible causes.

Note that neither Arc nor Q uses memory as emphatically as Manifold, so they're not really testing issues that might arise with memory or storage when Manifold reaches out to use those resources. Their results don't prove possible hardware issues are not the problem.

danb


1,970 post(s)
#18-Nov-22 20:19

I am guessing just me then. I will go back and have a closer look at the source data and see if the problem lies there.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri

7,115 post(s)
#19-Nov-22 06:39

have a closer look at the source data and see if the problem lies there.

Respectfully, I think it would be more productive first to consider what are known to be frequent causes of that error. Consider and exclude those first, and only then move on to less likely causes of the error.

I suggest starting with the following:

1. If you would like to tap into the collective expertise of other users, report details on what you are doing in this thread.

- What is the build you are working with? What Windows OS are you using?

- Is any network share in any way a part of the system you are using?

- What is the size of the source data you are working with, for example, how big the project is when stored as a .map file with only the vector layer you are using, and how many records there are in that layer?

- What, specifically, is the intersection operation you are doing? If it is a query, what is the query text?

2. Create a small subset of the source data. For example, if the source data you are using is 200 GB, create a drawing from it that is only 50 MB or 100 MB. What happens when you do the exact same operation on that subset?

3. Do you have a different machine to use that is the same as the one with the problem? If so, what happens when you try the same operation with the same data on that different machine?

artlembo


3,285 post(s)
#19-Nov-22 15:00

Did you attempt to perform a “clean” on the data? I think when you bring the data into a Geodatabase, ESRI does that automatically. Maybe give a try to use the clean transform first.

tjhb
10,017 post(s)
#20-Nov-22 01:11

I am guessing not just you Dan. I haven’t seen it, but haven’t tried.

I would guess that Art’s recommendation to normalize would “fix” the problem—but it would remain v interesting even so.

If you want someone to give an ~exact~ second try (even down to 5950X—what nice taste you have!), just let me know.

tjhb
10,017 post(s)
#20-Nov-22 04:21

At the same time, Dimitri’s example reminds me of the machine I had that thankfully was eventually stolen from my car in the night.

I had thought that the ~infinite spikes and pits I had been troubleshooting for weeks were something to do with my primitive code in Manifold.

I had tested everything (yes including RAM, exhaustively, with selective then complete replacement), without luck.

Only the theft of the box from my car finally identified and fixed the problem.

It can only have been the CPU. (Hard drives?—these were all spinning drives, and the errors though random were repeatable, and always on the same scale, so no.)

CPU errors are vanishingly rare, but I will always remember that vanishingly rare is not nothing—with thanks to the thief, but no apology to the sorry buyer.

adamw


10,239 post(s)
#22-Nov-22 11:44

Topology overlays have several limitations specific to them. For example, a key portion of topology overlays is currently single-threaded. There is also a limit on the amount of data you can process. For example, we are using a specialized data structure and the number of intermediate data items (roughly speaking, segments) has to fit into something like 30 bits. The process has several other data items with similar limits. Errors like the one shown in the first post happen when topology overlays hit one of such limits.

I checked and your case is still being looked at. There is no clear indication of any bug, but no indication of there being no issues and the process simply hitting one of the limits either. I will try to ask around and report back. But in general, topology overlays currently have (smaller than usual for 9) limits on the amount of data they can handle, that's the key issue that has to be resolved.

artlembo


3,285 post(s)
#22-Nov-22 14:59

are these limitations only in the GeomOverlayTopology.... functions, or all functions that handle geometric intersections like GeomClip?

adamw


10,239 post(s)
#23-Nov-22 09:04

Only in GeomOverlayTopologyXxx. Other geometry functions also have some limits (and some we would like to relax, eg, we'd like an area in GeomUnionAreas to be able to go above 2 GB in size at least temporarily), but they are different.

danb


1,970 post(s)
#22-Nov-22 20:17

Thanks for your interest in this all and for the great explanation Adam. I think I originally reported it as an 'issue' as I wasn't sure if it was a bug, though on the tin and without the inside knowledge, I blindly assumed that if I ran the tool on any data, it would produce the result I was after. This is typically my experience with Manifold 9 and it is extremely unusual for anything to fail hence sending it in.

The slightly odd thing with this intersection is that it is part of a process and hence I have done this same identity overlay before with earlier builds of Manifold 9 on the same machine without issue.

I then experienced the same error message on another large intersection which prompted me to post this thread. I have been a bit flat tack of late which is why I haven't responded here earlier, but I have been normalizing topology to see if that helps and it seems that one of the times I saw this message (not the one I sent to tech) was the result of some unknown issues with the source data.

It still seems odd that when I export the data from Manifold 9 latest (even with topology cleaned) to a geopackage and try to complete the intersection (as a union) in QGIS, that it is reporting a bunch of invalid geometries (see attached image), but this might be a separate issue? I will keep looking.

If and when the topology overlay functions are reworked, can I suggest that the option to preserve overlaps is included? Much of my work is with surface water catchments and their sub-catchments which naturally overlap each other. At the moment when I am undertaking intersections with catchment layers, I have to defer to Arc which in this case correctly preserves the overlapping parts of the catchment in the overlay and result.

Thanks all for your interest and offers and while I think of it, the brown areas in the attached image are where QGIS has rejected and skipped the geometry in the source (brown) so they are missing in the result (pink).

Attachments:
Clipboard-1.png


Landsystems Ltd ... Know your land | www.landsystems.co.nz

adamw


10,239 post(s)
#23-Nov-22 09:09

Hear you on overlaps, I will make a wishlist item.

Regarding cleaning geometry data, then exporting it to GPKG, and then getting error messages in QGIS - could you either post a small example here or send it to tech support? Because even if the issue with topology overlays is related, these two things could perhaps be fixed separately (and likely should, because that's safest). No fields needed, just the geometry.

danb


1,970 post(s)
#24-Nov-22 01:03

Here you go. Thanks Adam

The QGIS union log lists which geometries in the Manifold created Geo package were deemed invalid and hence skipped.

The image helps to show these visually i.e., the darker orange ones.

I was unioning the layer in the Geo package with the Landcover Database layer already sent to tech. The layer in the Geo package was cleaned using the transform in the latest cutting-edge build of Manifold 9 with a tolerance of zero.

Many thanks for your interest in this.

Attachments:
POL_2018_REGIONAL_AUTH_NZ_CLEAN.7z
QGIS Union Log.txt
QGIS Union Result.png


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.