Subscribe to this thread
Home - General / All posts - MF 8.x - problem with precisions
andrej
105 post(s)
#31-Aug-21 10:01

Manifold 8.x. I have a problem, namely when I want to export a local component to shp format, it always exports coordinates in a format that has as many as 8 decimal places, regardless of the fact that I fix Properties -> Precisions at 0.01 m. As a result, the state body rejects the prepared geodetic study. They require centimeter accuracy and not a thousandth of a millimeter. Is it possible that this is a bug in MF 8.x? Please help.


...Don't take life too seriously . No one gets out alive! ...

ColinD

2,038 post(s)
#31-Aug-21 13:00

You can format x/y lat/lon fields in the table. Right click the heading and select format.


Aussie Nature Shots

adamw


9,956 post(s)
#31-Aug-21 13:15

The value in Properties - Precision controls how far coordinates must be from each other to be considered different during vector operations. It sounds like you need something else - the coordinates to be rounded to the specific number of decimals. If so, try Drawing - Orthogonalize (Scope = [All Objects], Steps = 0.01 / 0.01, Offsets = 0 / 0, Unit = meter).

andrej
105 post(s)
#31-Aug-21 13:51

This is a mistake they send me back. As I wrote before, I only need two decimal places. The only strange thing is that I encountered this error for the first time since using MF 8.x. - It has never appeared before, despite the fact that I have created and successfully submitted many layers in shp format. Exports have always exported only two decimal places - now 8 decimal places at a time. And I don't know where the problem is or why it suddenly arose.

C14640 Number of fraction digits overflow at (503037,559996338, 131501,137712335)

C14640 Number of fraction digits overflow at (503042,549996331, 131503,147712336)

C14640 Number of fraction digits overflow at (503050,129996322, 131508,157712338)

C14640 Number of fraction digits overflow at (503056,909996314, 131512,037712338)

C14640 Number of fraction digits overflow at (503063,999996306, 131514,857712339)


...Don't take life too seriously . No one gets out alive! ...

andrej
105 post(s)
#05-May-22 05:06

How can I set by default that all coordinates will always be just two decimal places. Above all, I don’t know why this has changed in Manifold. All coordinates always had only two decimal places. I’m also thinking that from now until the end of time, I’ll never need more than two decimal places. Where and how can I set this up? Thanks for the help.


...Don't take life too seriously . No one gets out alive! ...

vincent

1,954 post(s)
#05-May-22 15:03

After setting precision, if you run Normalize Metric, does it solve the problem ?

andrej
105 post(s)
#06-May-22 10:35

Adam has already given a partially good answer to my problem. I still have a problem that every time I export a shp file I have to perform the Ortogonize operation beforehand, as suggested by Adam. In the heat of battle, I sometimes forget to carry out the operation Orthogonize, I send the data to the summary cadastre at the state level - in the end, the study is rejected because the coordinates have nine decimal places. So my question is "How do I set the default number of decimal places to just two decimal places?" So I set the number of decimal places once and the setting applies to all my MF projects! Thanks for the reply.


...Don't take life too seriously . No one gets out alive! ...

tjhb

9,970 post(s)
#06-May-22 13:18

You can't. It has never been possible.

The internal datatype respects binary floating-point precision as far as possible.

The shapefile standard does the same, but only up to 6 decimal digits. Manifold follows that standard, when necessary.

You could write a custom exporter that did not respect inherent precision, but met arbitrary legislative requirements. Wouldn't that be fun.

Your best bet is to challenge any stupid legislative requirements you might face.

tjhb

9,970 post(s)
#06-May-22 13:43

(Even after orthogonalization, as adamw suggested, you can't restrict the number of places in decimal format for all numbers, because they are stored in binary. There is simply no way, besides rounding or truncation on export, which is data loss. Even if achieved, it will expose you or the client to arbitary topological errors. Just say no.)

adamw


9,956 post(s)
#07-May-22 11:44

As Tim says, there's no UI option to force the export to round coordinate values to some specific number of decimal digits. There is a general tool for that: Orthogonalize, but there's no option to call that tool from the export.

If you frequently need to export to SHP with rounded coordinate values, you can write a script to help. Method 1: write a script that would make a copy of the opened component, round the coordinate values, export the result, then delete the copy. Method 2: write a script that would take a folder of SHP files, read each, round the coordinate values, export back, then you could export normally and periodically run the script to force the rounding. Method 3: write a script that would simply run Orthogonalize on the opened component, automatically rounding the coordinate values for that component to two decimal digits, with no further user input so that this only consumes a single click and is not a big hassle. Etc.

If I had to choose, I would go with method 2, because it can fix the SHP files after you exported them even if you no longer remember which MAP files they came from. This can also be packaged into a separate EXE that you would run outside of Manifold.

A question - what tool do the C14640 errors above come from? Specifically, does that tool work with SHP files directly or does it pass SHP through some kind of printing and work with the resulting text (which seems possible)? Because if it is the latter, then perhaps you can apply the rounding when the coordinate values are being printed, that would be much better than applying it to the raw data.

tjhb

9,970 post(s)
#08-May-22 15:34

there's no UI option to force the export to round coordinate values to some specific number of decimal digits. There is a general tool for that: Orthogonalize, but there's no option to call that tool from the export.

I am not sure. (Same worry with methods 1 to 3.)

Let’s say you orthogonalize to 2 decimal digits (or any number of decimal digits).

Fine, but in any case the representation of the result will always be in binary.

In a following export to shapefile, the stored binary result will be translated back to decimal (effectively as text, with 6 decimal places, though this limitation does not usually matter and is not the main point).

When the exported result is retranslated back to binary (on import to any platform), it may not map back exactly to the orthogonalized result, because of binary/decimal representation differences.

It will only reliably remap to orthogonalized values, if somewhere in both the export and import pipelines, there is a pragmatic that maps the most succinct decimal (text) representation to the closest binary mantissa and exponent (and especially vice versa).

That might happen, it might even be the case generally, I don’t know. Iff so, orthogonalization works for this export “issue” (which I think is anyway a false issue).

The same applies a fortiori to rounding.

tjhb

9,970 post(s)
#08-May-22 15:55

Again I think it is incumbent on us as GIS prefessionals to strenuously object to laws and regulations which we know to be dumb.

Insistence on representing geographic coordinates to only 2 decimal places in is really only marginally different from Dimitri’s often-cited case of legislating that pi=3.

adamw


9,956 post(s)
#12-May-22 15:46

It will only reliably remap to orthogonalized values, if somewhere in both the export and import pipelines, there is a pragmatic that maps the most succinct decimal (text) representation to the closest binary mantissa and exponent (and especially vice versa).

Yes, with a note -- parsing always behaved that way in that parsing a rounded value was always producing a floating-point value closest to the string representation, we are only talking about printing.

Given proper printing functions, rounding a floating-point value to X decimals and then printing it should produce a string with no more than X decimals. That's because a 'proper' printing function is such that it prints the shortest possible *decimal* string that translates back to the exact same floating-point value.

For example, when we round, say, pi, to 2 decimals, the exact floating-point value that we get is something like 3.14000000000000012434... (proof: StringFormatNumber(RoundDecs(PI, 2), 'f20', '')). But this does not matter because if we just print 3.14, that, when parsed back into a floating-point value, produces the exact same floating-point value (proof: StringFormatNumber(CAST('3.14' AS FLOAT64), 'f20', '')).

Rounding to decimal fractions is safe that way, because printing, similarly to rounding to decimals, is also concerned with the shortest form in the decimal notation. Rounding to non-decimal fractions (eg, the infamous 1/3) is unsafe.

You can verify that rounding to decimal fractions does not produce extra digits during printing by running something like:

--SQL9

SELECT Max(StringLength(CAST(value/10000 AS NVARCHAR)) - 2) -- exclude '0.'

  FROM CALL ValueSequence(0, 10000, 1);

The above produces all 4-digit fractions between 0 and 1, prints them, counts how many decimals are in the result, then finds the maximum number of decimals -- the result is 4, as expected (might be smaller, because values like '0.1000' are printed as '0.1' = one decimal, but not larger than 4).

So, if the function used to print floating-point values actually prints the shortest possible decimal string that translates back to the value, the printed values should not have any extra digits. If the software package that performs the printing is reasonably modern, it likely prints that way. That became the default way to print some time ago.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.