Recently, I've been working with large datasets of point features and I've come across an issue. When I use an attribute to highlight some of the points with a larger or bolder symbol, I would expect to see all of those features pop out amid the remaining, smaller dots. But I'm finding that only a subset of those important points appear while others are hidden, and when I zoom in and out of the data, those more significant points pop in and out of view seemingly randomly until they all display properly at a large scale.

I'm attaching an example of ranked hills and mountains, styled using the PEAKS_80KM field. The lower the number, the greater its relative importance. Watch the Alps carefully as you zoom in and out, and see if the larger circles pop in and out of view. Panning at the same zoom level also makes features disappear and reappear sometimes.

It seems like the data is being sampled for display with each view, but I would like to be able to see all points at all times. Has anyone else run into this?

Attachments:
Peaks Western Europe.mxb

In 9, Z-order within a drawing is effectively random. The engine fetches and paints with priority given only to speed.

Being able to define Z-order within a drawing (with a resulting speed reduction in most cases) has long been on the list. Who knows with what priority.

The only way currently is to split data into separate layers. Usually that's not a big deal. In fact it is an elegant solution. Build display drawings from live queries, each having a filter. Layer order is respected in rendering.

Thanks for your response. Splitting the data into separate layers is a good workaround, but I'm still puzzled as to why this is needed. So Z-order not only determines which objects are rendered first, but also whether an object renders at all? I really like the speed of Manifold 9, but I assumed it was rendering all the objects in the visible layers, perhaps with some simplification, not displaying an arbitrary subset of the data.

I'd like to get back to this subject of point features appearing, disappearing, and apparently not rendering at all in certain circumstances in Manifold 9.

I think it would be nice for users to be able to control the trade-off between rendering speed on the one hand and rendering accuracy on the other. There are times where speed is best and it isn't a problem if the objects shown on the screen are a mere sample of the whole dataset. But there are many other times when I need every data point to appear, even if there are lots of them and rendering takes longer. Whether it's visually analyzing data patterns or creating a finished map, it's essential to know when the data has been completely and accurately rendered and when it hasn't.

Manifold 8 has a checkbox to enable smoothing large vector objects for performance, it's under Tools > Options > Rendering. Something similar in M9 that applies to points, lines, and areas would be very useful, in my opinion, so I'll be contacting Manifold support with this suggestion.

Anyone else have an opinion about this?

I totally support you on that!

Let's the user have the control on Speed VS Precision in the displaying of objects in M9.

Thanks, Corentin.

This is a significant topic that is worth discussion. Thanks for raising it! Apologies in advance for the length of this post...

when I zoom in and out of the data, those more significant points pop in and out of view seemingly randomly until they all display properly at a large scale.

and

I think it would be nice for users to be able to control the trade-off between rendering speed on the one hand and rendering accuracy on the other.

Rendering speed isn't the issue. Manifold renders everything basically instantly, especially in small data like this, so a desire for speed does not cause the visual effects described.

"Accuracy" isn't the issue either. Using that word is compounding an error in the first quote above.

The error is "those more significant points." That's an error because the designer of the map has told Manifold all of the points are equally significant. Stating that some are more significant than others is the old "do what I mean, not what I told you to do" bit.

The order of layers in a map specifies which objects are shown above other objects. (See the Maps topic for example). If you want objects to be shown above other objects, put them in a higher layer and they will, without fail, be shown above objects in lower layers.

That is a simple system that is easy for new users to grasp and is easy to operate with simple controls. When a map has just a few layers you can drag layer tabs in a map left and right to order them as you like. When a map has many layers you can use the Layers pane to manage the ordering of very many layers in fast and sophisticated ways, including moving many of them at once higher and lower in the display stack or turning many logically related layers off and on at the same time.

Putting records (objects) into the same layer tells Manifold they all have equal importance. The system doesn't know the map designer really meant some of those points were more important and it won't overrule what the designer told it to do. It's going to do its best to provide a rendering that captures a representative view of the layer considering all objects in it without simply drawing a black blob that is visually meaningless.

That's where the misuse of "accuracy" comes in, since usually it is not possible to show *all* of the objects in a layer at the same time, for example, if there are not enough pixels in a monitor or in a region of the monitor that the user has told the system to use (via a zoom command). The sample data set provided in this thread is a good example.

The sample data has 108,518 points in it. Zoom out to where the data set is the size of one square inch on the monitor and there are only about 9,200 pixels available in that square inch. But there are over ten times more objects to show than there are pixels in the designated region of the monitor. It's physically impossible to show all the objects "accurately" at that zoom level, if "accurate" is supposed to mean displaying each and every object.

The sample data also illustrates a second effect that makes it impossible in most cases to display every object: the styles used in the map are a mix of large symbols and small symbols, ranging from 24 points to 2 points in size. Depending on the zoom level, a single big symbol could cover up thousands of small symbols if it is drawn higher. Likewise, thousands of small symbols could collectively completely cover a single big symbol if the small symbols are drawn above the big symbol.

Neither case would be "accurate" if by "accurate" is meant a visual display that shows all objects, or which might not give somebody the wrong impression of data that is located in a given region.

For example, if a person sees one big circle surrounded by many tiny dots he or she might think, "wow... only one big mountain in that region and no small ones". Conversely, a solid carpet of tiny dots might give the impression, "nope... no big mountains here, just lots of small ones...".

If you construct and style a map as has been done in the example, in most regions at most zoom levels there is physically no way to avoid bigger symbols covering up smaller ones or smaller symbols covering up bigger ones. Specifying Z order within the same layer using a field is not a silver bullet to make it more "accurate" in the sense of either showing all objects or not giving false impressions, because it would just guarantee that in many zoom settings most of the data would be hidden, with up to 99.9999% of the data being hidden when zoomed far out. How can it be said that a display which fails to display 99% of the data is "accurate"?

On the contrary, one could argue that the most "accurate" rendering is a display which shows the *maximum* number of objects possible. That would prioritize small symbols, because more of them can be rendered into whatever screen real estate is available than just a few big symbols.

Neither of the above two extremes or the many gray areas in between can be handled by a "one size fits all" approach to rendering and none of them is really about "accuracy." All of the cases are about different choices in terms of how a given data set usefully may be shown to users even though in no case is is possible to show all of the data all of the time.

Some of that is a consequence of the choice of style made by the map designer. For example, if you want to show more points in a layer without big symbols shoving many smaller symbols out of the way, well, use small symbols for all of them and indicate bigger mountains by color or symbol shape. Dealing with such tradeoffs is a routine part of cartographic design.

That's why Manifold provides a wide variety of controls that allow GIS people to create a map which conveys the essence of what they want to convey despite the certain knowledge that users will pan and zoom the data to create radically different views. Controls like the ability to create multiple visual layers from the same table (data set) with individual settings for zoom ranges, styles, opacity and so on are essential tools for the cartographical task of creating a map that even at different zoom ranges does a good job of conveying the message the map's designer wants to convey. That can be time consuming, as good symbology in cartography is often very much zoom related. Very often a good design requires skilled use of zoom ranges, opacity and many different options in style.

If a designer doesn't use those tools but instead puts everything into a single layer, the system starts with the "OK, everything here has equal importance" standard and tries to paint a layer that is visually digestible. That's where displaying a representative view of the layer without simply drawing a meaningless black blob comes in.

If that's done wrong, you get stuff like this, which some web map rendering engine (not Manifold) produced. Note the region of solid black where too many symbols created a meaningless black blob.

That's why it makes sense to clip some objects and not to attempt to show all of them if the objective is to provide a representative view of data and not a meaningless black blob. Clipping can enhance speed, of course, but really it is about avoiding visually meaningless mush.

When rendering an individual layer, starting with the "all records in a layer have equal importance" essence of a layer, Manifold uses a variety of clipping algorithms on the fly to render the points, lines and areas in that layer. The result usually is a reasonable representation of the overall essence of a layer despite wide variations in zoom level. The sample .mxb provided shows that because despite variations in zoom users can see the data has a mix of bigger and smaller mountains, and in most cases can get at least a visual impression of the relative proportion between numbers of bigger and smaller mountains.

If you want to get more control over display order or clipping, the first step is to use layers to do so and to use zoom ranges to turn "smaller" object layers on/off as you zoom in/out. That actually works out much better than Z ordering based on a field, because avoiding ugly displays like the black blob effect above requires clipping objects, which Z ordering by itself does not do.

An example of that effect in action is the base map that is used in the Eclipses web map demo at https://manifold.net/ims9/ - as you use the mouse wheel or zoom buttons to zoom further in (try Europe) you can see how at first only bigger cities appear and then more and more smaller cities. The effect is accomplished with only four layers: the original, big populated places drawing was split out into four drawings with populations for big cities, cities, towns and places, with those four layers added to the map with different min/max zoom display ranges.

That's easy to do without creating drawings from queries, but if you prefer to keep all of your data in the same table you could create multiple drawings from queries against that table to select only part of the data for each drawing.

Using layers is conceptually simpler and easier for beginners than Z ordering from fields on a per object basis. Drag a layer tab left or right, with no need to use a sequence of select and transform steps to alter Z ordering numbers in a field. Move a populated place object from one layer to another by simply cutting it from one layer and pasting it into the other.

Could a Z order option by field be useful within a layer? Sure, but not for what seems to be the main complaint here, which seems to be about clipping more than Z order. How Z order interacts with clipping is a complex discussion that one runs into with label engines, and it's not a case where simple rules guarantee either "accurate" (whatever that is intended to mean) displays or comprehensible displays. For example, if lower Z field value objects can never clip higher Z field objects that just guarantees only higher Z field objects will appear in the display at zoom levels where their size dominates. But they'll do that anyway given Z ordering and result in 99% of the data being hidden. In edge cases where there is a continuous range of Z ordering because it was taken from some field like the height of a mountain (inevitably what happens, since nobody is going to manually Z order 100K+ objects), there will be effects that aren't really as controllable as the use of layers because they involve slight differences between Z values in too many objects to resolve manually.

You can deal with that by creating a new Z field that quantizes a continuous range of Z values into a limited number of bins, but if you're going to do that it's easier to just select based on a range of Z to create a limited number of layers, like the four layers for populated places in the eclipses map.

Z ordering based on a field is one of those things that's emerged based on a desire to achieve a "one size fits all" mechanism in GIS packages like command-line driven web servers that don't have a good, interactive layers interface like Manifold. But sometimes the "pick a Z field and then curse the tool when it creates a blob", like the image posted above, shows the limitations of the approach.

Just saying, there are many tradeoffs in terms of different approaches to providing a set of controls which make it easier for a map designer to create a map that conveys the impression desired despite radically varying zoom ranges. My advice is to master the tools and controls which already exist in Manifold, as they can achieve a wide range of effects based on simple approaches that are easy to learn and fast to apply. If those controls are not enough, suggest something additional. But consider carefully if that something additional does in fact fix a limitation.

Attachments:
blob.png

Sure the layer approach is conceptually easy and works but as the project gets complicated with lots of layers it quickly becomes tedious and messy. A map may contain 10 layers that need 3 levels of z ordering, as well as different labeling. That means potentially over 50 layers that floods the project pane and makes modifying the whole set up tedious. Plus there is the zoom layering display setting.

I think Manifold could make a lot of people happy if there was some sort of dialogue associated with components of a table/query that allowed setting up the z ordering, labeling, and zoom level so that it can all be seen together and not require accessing each component to make adjustments. To keep the project pane tidy, it would be nice if it had the ability to stack most everything component under a table that references it.

Thank you very much for your thorough post, Dimitri. I really appreciate the time you've taken to address this topic.

You're right to point out that "accuracy" isn't the best word here, I think "completeness" would be better. For what I'm doing, I'm most interested in the patterns of spatial distribution and I need to take all data points into account to do that. As you mention, creating a separate layer for the objects to be highlighted is good practice for a finished map, but it would be cumbersome to do that each time you want to look at the data.

I use Illustrator/Mapublisher for cartography, but Manifold 8 and 9 are the best tools I've found for data transformation and analysis. Currently, I'm working with huge datasets of cities and mountains in order to pre-select some for display at specific scales, just like the cities shown in the web map demo you mentioned. To do that, I need to take into account not only population, but also geographic distribution in order to fill the map somewhat evenly with cities and leave ample room to label each one (I've explored this subject in the forum several times over the years).

The first attachment is a concrete example showing cities and towns along the East Coast rendered in QGIS. All the red and some of the orange and yellow cities would be candidates for labeling, while the others would be hidden from view at this scale. Transparency prevents the larger circles from completely covering the little ones and no Z order has been applied. This isn't a finished map, of course, but rather a quick glance at the data. No data seems to have been clipped from the image, which contains 36,000 points.

The second attachment shows the same data and map extent as rendered in Manifold 9. While it looks almost the same at first glance, if you look at it closely, you'll see that only five red circles appear in Maryland and Delaware in the Manifold image, while ten appear in the complete data render in QGIS. In the QGIS image, you can easily see a dense cluster of small dots from Maryland up to Trenton, but this is not really visible from the Manifold screenshot.

The bottom line is that the Manifold user is not necessarily seeing all the data points that fall within the extent of their map and this could affect the conclusions they draw. I didn't realize this at first and started scratching my head and tinkering with my queries. I've never noticed this behavior with the other GIS software I use, so it didn't even occur to me that some data was being clipped from view. I don't remember seeing anything about this in the user manual either, but please point it out if I'm mistaken.

The way it is now, Manifold is selecting which data points appear and which should be hidden, and that is just fine in some circumstances. But I think it would be much better to let the user choose whether they want a simplified/clipped view of the data or the whole thing. As I mentioned before, Manifold 8 lets users turn on and off smoothing of large vector objects and I think something like that for areas, lines, and points would be very valuable in Manifold 9.

Attachments:
1 - QGIS Eastern Seaboard.png
2 - Manifold Eastern Seaboard.png

To my surprise I could not at first replicate the culling behavior. Then I copied data from Postgresql to manifold project and got culled view. So, can I recommend you load your data to Postgresql ?

Attachments:
m9_native_cull.png
m9_pg_no_cull.png

I thought, that the way to do "priority rendering" would be something like this.

• We have table/drawing with *many* objects.
• We want some of them to "stick out more"
• We create or use an existing attribute that represents our need.
• We create a query that selects above our threshold x

--sql

SELECT * FROM [many_objects] WHERE [z_order] > x;

• We create drawing on that query and use that drawing on top of main drawing.

There's one catch though.

• We have to use query with TableCacheIndexGeoms

-- $manifold$

TABLE CALL TableCacheIndexGeoms(

 (

 SELECT * 

 FROM [many_objects] 

 WHERE [z_order] > x

 )

 , FALSE

);

Otherwise, culling still occurs.

Attachments:
m9_query_culled_wo_cache.png

Thanks rk for taking the time to tinker with a large dataset and posting the results. It's interesting that with Postgresql, you were able to get results in M9 without any clipping. I'm not familiar with Postgres at all, but I see the Manifold user manual has a number of pages on it to get me started.

Great illustrations!

Let's take a closer look at what you're doing, to see how to accomplish what you want...

> I'm most interested in the patterns of spatial distribution and I need to take all data points into account to do that.

Well, in any package there's a difference between a quick look to guide further work and then the more detailed work that follows. For example, you're not going to be able to visually distinguish 36,000 points, so a quick look isn't literally about *all* data points. It's about a useful summary.

A useful summary is what you show in the two illustrations, neither of which displays all data points. Both Q and 9 provide summary views, which, as you note, are so similar that "it almost looks the same at first glance."

> creating a separate layer for the objects to be highlighted is good practice for a finished map, but it would be cumbersome to do that each time you want to look at the data.

If all you want is a quick look, you don't have to create separate layers. Just use what the package (either Q or 9) provides by default and then take a few seconds to pan and zoom a bit to decide what you want to do in a finished map.

If you want, you can refine the quick look with one or two minute's work - easy to do without creating a lot of layers. After all, it's not a binary choice between "create many layers" or "don't use any layers". You can create just two layers if you want.

The sample maps posted are a good example where just two layers can help refine the quick look without investing a whole lot of time. For example, in the five bins you've used in the sample map 99.6% of the objects are the tiny dots. That's just one layer, and splitting that single layer into two layers, one layer for many tiny dots and another layer for the few big dots is fast and easy.

I split your sample "Peaks" layer into two layers, including creating the drawings, in a minute or two. That's easy enough to do with the Select pane, but I did it even quicker with a query:

SELECT * INTO [Peaks 999 Table] FROM [Peaks W Europe] WHERE [PEAKS_80KM] = 999;

I made a "999" drawing from the resulting table, and then I copied/pasted the query to modify it to

SELECT * INTO [Peaks others Table] FROM [Peaks W Europe] WHERE [PEAKS_80KM] < 999;

to make an "others" drawing. I used "INTO" to avoid using TABLE CALL TableCacheIndexGeoms(). Had I wanted to create a "final" map, I could have created three more layers in another minute or two to handle all five "bins" in the map. I could then play with Z order, transparency, and such without thinning between layers.

The result of two layers in a map:

But I didn't have to do any of that for a quick look. For a quick view to guide subsequent analytics or creation of a final cartographic arrangement, I wouldn't have bothered to even spend the minute or two to split the map into two layers. I just zoom into a region or two of interest to see what's going on, for example, to see what objects are in dense clumps, to decide if I want to split it into four or five layers.

I don't think that quick views, like the two sample images comparing the Q and 9 display, bear much on the techniques one would use to create a final work product. Neither is a final work product and both "hide" some data points while displaying others. In both cases that's not a big deal because a bit of panning and zooming in a matter of seconds reveals what one might want to do in a final work product that will help avoid any errors of interpretation by an unskilled user.

In fairness, the following applies to both Q and Manifold: "The bottom line is that the Manifold user is not necessarily seeing all the data points that fall within the extent of their map and this could affect the conclusions they draw." - That's because the Q user also is not necessarily seeing all the data points.

It's not a case where one display shows "all" points and the other doesn't. They both are interpolations, just done differently. Here's a case where the Manifold display shows points that are either hidden or very hard to see in the Q display:

Note the very visible orange dots marked by blue arrows in the Manifold display that are not discernable in the Q display.

If you zoom in and look carefully you see that they are there in the Q display as well, but are difficult to see:

In both cases some dots are visible in one display that aren't in the other. It's just a different set of dots.

The above display also shows a consequence of the mass of 999 dots which almost hide one of the big red dots. Q's willingness to clump dots that way is what sometimes causes a "black blob" effect in Q displays which Manifold avoids. In this case, you mention that you're happy with that gray blob effect because it helps you see denser grouping of dots in some places. But that's a happy accident of the particular zoom used for the two displays. A side effect is that you don't know if the denser blob is hiding some dot below it. You really need to zoom in to such clusters to find out, but if you do that you'll also see the relative density (which Manifold also shows, just less emphatically than Q in this case) better of such denser clusters between Maryland and Trenton.

In either case, if emphasized cluster density is the objective for the final map, it's probably better to display the data as a raster in which contrast can be enhanced.

I don't think that matters in either case because the two views are both quick looks for the map designer, not a final work product for unskilled users.

Anyway, my point is that there is no one rendering method that fits all needs. There are strong benefits for visual insight from thinning out the objects rendered at various zoom levels. That has downsides sometimes too, of course, but then so does *not* thinning out objects at various zoom levels. If you don't thin objects you end up hiding some, and as the example posted shows, not all options (like transparency) can fix all the downsides of that in all situations. In both cases it's just a question of how to manage downsides from either approach in some specific design/zoom/data situation.

The Manifold approach to strike a balance between effective workflow and managing any downsides is to try to render something that works well in most cases and which avoids blobs. For greater refinement, the toolset is designed to use layers and to make it easier to use the power of layers. That's not the only possible approach, of course, but it does leverage the power of other features in the Manifold toolset.

My own experience indicates that turning off thinning sounds good until you try it, and then you may realize that it's usually a viable option in cases where the data already is so thin you don't need it. Otherwise, you get the blob effect. You can see that in action with labels:

The above illustration has thinning ("culling") turned off.

Just saying, using layers you can get really super effects in a final product, and even without using them, or using them just a tiny bit, you can get very effective browsing for a quick look. You can also use techniques like rk mentions, although that does require more skill. If the above is not enough and something else would be more helpful in your view, that's OK too and definitely what suggestions are for.

Attachments:
compare_01.png
compare_02.png
no thinning.png
two_layers.png

Thanks for your help, Dimitri, and sorry for taking so long to respond. Using two layers is a good way to solve the problem of drawing attention to a specific subset of the data for a quick look. The colors and symbols are only intended to help me see whether the most important cities or peaks are evenly distributed across the map. In a finished map, those red locations would be triangular peak symbols with labels, etc. and the rest of the points would be hidden or deleted. Now that I know Manifold clips large point layers, I'll change my workflow to account for that.

The suggestion of zooming and panning the map as a quick way of getting a sense of what is going on brings me back to the second reason for my post: why does Manifold clip the points (you've already given good explanations about that), what users think of the clipping behavior, and whether anything can be improved in that area. When zooming in on the data, some of those colored circles pop in and out of view at each zoom level until the scale is large enough there is probably no clipping. Even if I keep the scale the same and simply pan left and right, some of the exact same colored points appear and disappear from view. Since Manifold doesn't provide any feedback about how many or what percentage of the points have been rendered each time, how does the user know if they're getting a complete view of the data or whether some has been clipped? For that, a line in the Info pane such as "28,400 of 36,000 points within the window extent are displayed" might be helpful.

You mention that the QGIS screenshot in my post above doesn't show all the data points. I can't count them all to be completely certain, of course, but let's use the colored circles as a sample of the data for a moment. There are 50 of them in Pennsylvania on the QGIS image (Manifold shows only 37), and when I intersect the points drawing with Pennsylvania as the overlay layer, I get a total of 50 records with values that correspond with the colored circles. In other words, none of those points have been clipped from view. In other areas of the map, I've found that the size of the symbols sometimes make it hard to see points underneath, but I haven't found evidence that QGIS is rendering only a subset of the data in the view. I loaded the same data into Mapublisher and Global Mapper and got the same result - areas of densely packed points in Maryland and no clipping apparent - though Global Mapper doesn't allow for transparency on point layers, so smaller ones are hidden from view in some places. On that subject, you're right to point out that some dots appear more clearly in the Manifold display. But I also think there's an important difference between points that are hard to see because of the symbology or transparency used and points that are entirely clipped from view. This clipping behavior is something I've only run into with Manifold.

Finally, I totally agree with you that large blobs of data or labels are usually useless and thinning can be very helpful. But as I've explained, there are times where it would be good to have the option of showing all the data. Do you see any drawbacks with making clipping and simplification an option the user can turn on and off? How about a line in the Info pane that reports how many objects have been clipped from view?

In other words, none of those points have been clipped from view.

It's not that they're clipped from view in Q, it is that they are hidden by other points. That's the issue... if you have many points, some will cover other points. If the complaint is that the visual display does not show *all* points it doesn't really matter if the reason all points are not shown is because some are covered by clumps or other points or because some are clipped to avoid a blobbed display. Either way, not all points are seen.

For example, your question could be written as:

Since Q doesn't provide any feedback about how many or what percentage of the points have been hidden by other points each time, how does the user know if they're getting a complete view of the data or whether some has been hidden?

Why not request that Q provides a report of how many objects have been hidden from view by other objects?

Whether objects are clipped or covering each other or just merged into blobs that's not an issue for most users, since people naturally understand that if they are looking at a road map of the entire United States or Europe or whatever on a computer screen they don't expect each and every road will be drawn. I don't think people expect a readout that gives what percent of the 22 million roads in the US can't be drawn at the current scale, or that such a readout would be helpful. I guess in some situations that might be helpful, but it seems those situations would be very few.

Do you see any drawbacks with making clipping and simplification an option the user can turn on and off?

Well, it's not a hot button for me so I don't really see much upside or downside either way.

There are zillions of small features, modifications, and options that are perfectly possible to do. What they all have in common is that none of them are free, either to implement or to deal with in the course of using the tool. They all take time to implement, time to document, time for new users to learn and time to deal with in the user interface. Stuff that seems perfectly harmless to add isn't totally harmless when enough of it piles up, like when you get packages with so many zillions of options they become a burden for normal users to deal with in terms of labyrinthian Options menu systems that are a headache to navigate.

One other thing that all those zillions of features that are perfectly possible to do have in common is that they compete with each other, and with bigger features as well. If there's more demand for something it bubbles up in priority until it gets done. The Suggestions web page has a good description of the other factors that go into prioritization, like whether something is hard or easy to do.

Clipping is a sophisticated mechanism. It's hard to do, which is one reason you don't see it in so many systems. It's not easy to thin on the fly from a mix of points, lines and areas and to do so adaptively for various zooms in a way that usually provides a good visual impression of the contents. It's also harder to do in a fully parallelized setting where many threads might be doing things with objects that pop into or out of existence, and it has to be done with awareness of many cache issues.

I don't know how the code works in Manifold, but based on the externals it seems a fair guess it's something that requires significant skill to be modified competently, to ensure just turning it off or on doesn't lead to unexpected impacts in various subsystems. I'd bet none of that is rocket science but I bet it would take more than a couple of hours to do right. Add to that the usual overhead for updating options dialogs, ensuring everything gets stored/remembered right in various different Windows systems and all that usual routine jazz.

If I were to suggest it, I'd suggest not implementing it as a Tools - Options setting but rather the same way overlaps/no overlaps are implemented for labels, as an overlaps setting in the Layers pane. Then it would be more orthogonal, the same control and same options, just extended to vector layers in addition to labels layers. I'd guess that's a bit more work than just tossing in another checkbox in a modal dialog but I think it would be the right way.

So, no big deal I'd bet but probably enough work to where it has to compete with other features, thousands of them, people have requested.

How about a line in the Info pane that reports how many objects have been clipped from view?

My guess is that would be profoundly more difficult to implement and, frankly, not particularly useful, while confusing most people. But that's just my personal taste. If it were a priority for me, I'd just say add an option to clip/not clip.

This isn't the first time there's been a discussion like this, because a very analogous discussion rotated through the community about clipping labels. When enough people asked for an option to not clip labels, Manifold added that. What happened? Research indicates it's an extremely rarely used option, for the reason that the last illustration in my prior post makes obvious.

You can come up with a similar, unthinned illustration for vector points:

The above shows the sample Peaks W Europe drawing loaded in from a PostgreSQL data store, with no thinning. Lots of clumps from all those unthinned points and not a particularly appealing (or, to my taste, useful display - I liked the "two layer" example better). GIS packages usually have tools for hopefully creating a better display, like assigning Z priorities, transparency, and such, but those all require additional tinker time or even real work. Are they better or quicker than using Manifold layers, opacity, and such skillfully? Not to my taste, but of course other people's tastes can be different.

There's also the factor that if you really want unthinned displays, those can be done now in a matter of seconds. To create the unthinned illustration above I copied and pasted the Peaks W Europe drawing/table from the sample project into a PostgreSQL installation (see the topic for how to set one up). It took literally about ten seconds, and then another few seconds to drag and drop the layer from the PostgreSQL data source into the map. All the formatting was preserved, of course, so there was zero extra effort.

If it was me I'd just add a switch to turn thinning on/off if for no other reason than to make a point. But to get a feature ahead of other priorities if a) nobody in years has ever requested it and b) there is a quick workaround to do it, well, that's usually an uphill sell. My advice would be to send in a suggestion, making full use of the tips in the suggestions page, and to kick back and see what turns up over the course of time.

Attachments:
unthinned_peaks.png

The various examples in this thread make me believe that 9 has some sort of thinning filter turned on significantly too high and should turn it down. As mentioned, a layering scheme is the better way to handle what should get displayed. The layering scheme should be easy to set up and fine tune for many layers often found in decent maps, such as roads, towns, summits, rivers, lakes, places, parks, etc. Cascading style sheets come to mind as a method to put it all in one location, but Mfd should do better with some sort of user friendly window/dialog.

I'll submit a suggestion. Thanks siteatlas and others kicking the issue around.

The various examples in this thread make me believe that 9 has some sort of thinning filter turned on significantly too high and should turn it down.

This is something that I have long had a sense is going on, presumably as a tradeoff for rendering performance. It is demonstrable with las point cloud data.

I would definitely second a straightforward method to tune this filter to the task at hand accepting that this may result in a decrease in rendering performance.

Landsystems Ltd ... Know your land | www.landsystems.co.nz

I am not 100% sure of that behavior but I have the impression that if you select some records in a very large dataset they may not even been displayed by M9 in the drawing view for the reason we discussed earlier in this thread.

If a features would be implemented dealing with this behavior it should display all of what have been selected.

As a matter of fact it could be a way to deal with this behavior :

• no selection => priority to the performance over completeness of the display
• selection => total display of what is selected whatever the cost in performance. None selected objects may not be displayed in order to improve performance.

Thanks for your response, Dimitri. I think we've both explained our positions pretty clearly, so for my part, I'll just leave it at that. I've submitted a suggestion about this and I'd be interested in hearing other users' opinions, too. So by all means, keep up the discussion!