Hi all, I'm currently evaluating Manifold mainly for its ECW export function so that we can use the ECW with other construction surveying software but having some issues with the projection upon import. I have a GEOTIFF of about 60GB, which measures roughly 3.5Km x 3.5Km at its extremes and has been exported from Propeller Aero. It is on a near scale 1 local cartesian grid (UK OS HS2 Snakegrid) but when I import into Manifold I am getting a shift/rotate and small scale error. I've proved the GEOTIFF to be good by importing into other software such as QGIS and Global Mapper and checking against the many GCPs we have across the site. When I import to Manifold though, I can't seem to find an option to just use the projection as provided in the GEOTIFF, as can be found in Global Mapper etc? Can any of you shine some light on where I might be going wrong or as to how I can carry out a shift/rotate/scale within Manifold to get the image in the correct position? I know that I could use a WGS84 GEOTIFF which would most probably solve the projection issue, however when I export to ECW, this would not give a cartesian projection which is required by our other software? TIA.

This is a tough one to answer without some data to play with and 60GB is not trivial even in this modern time. Are there any world files or other associated metadata with the TIFF to look at? There have been recent refinements in Manifold's use of ECW as well as TIFF, make sure you have the most recent version. I am unaware of the Snakegrid.

I have the latest version. I can export the GEOTIFF in WGS84 which should import ok. However, any exports such as an ECW will then be in WGS84 unless Manifold has the option to export in HS2TN15?

Can you paste the output of the command

gdalinfo.exe the60gb.tiff

HS2 Survey Grid (https://epsg.io/9300) appears to be a system that needs a grid file for accurate transformation.

HS2TN15_NTv2.GSB

Do you have manifold's grid.dat file?

You could assign epsg 9300 to the imported file and check if grid transform becomes available when you attempt to reproject the image? I checked myself. No! For epsg 27700 it does, but not for 9300.

Though, if you are not changing the images coord system nor rely on Manifold's on the fly reprojection, it does not matter. If all you want is to import tiff and export ecw you could do it with Manifold. Just assign EPSG 9300 to the exported file.

We have the NTV2 transform for EPSG:9299 / EPSG:9300 (to EPSG:4258 = ETRS89, lat/lon), we are just missing the grid file.

Found this on another forum for importing into QGIS if any help?

PROJCS["HS2_Snake",

  GEOGCS["GCS_HS2SD",

    DATUM["D_HS2SD",

      SPHEROID["WGS_1984", 6378137.0, 298.257223563]],

      PRIMEM["Greenwich", 0.0],

      UNIT["Degree", 0.0174532925199433]],

  PROJECTION["Transverse_Mercator"],

    PARAMETER["False_Easting", 198873.0046],

    PARAMETER["False_Northing", 375064.3871],

    PARAMETER["Central_Meridian", -1.5],

    PARAMETER["Scale_Factor", 1.0],

    PARAMETER["Latitude_Of_Origin", 52.3],

    UNIT["Meter", 1.0]] 

For the

proj

+proj=tmerc +lat_0=52.3 +lon_0=-1.5 +k=1 +x_0=198873.0046 +y_0=375064.3871 +ellps=WGS

Are we talking about Manifold 8 or Manifold 9? Could you resize the 60 GB file, to, say, 1 MB, retaining coordinate system info, make sure the registration still has the same issues, and post the file here?

How exactly are you checking the registration -- are you looking at lat/lon of a set of control points? If so, the lat/lon values are going to be produced by converting from the coordinate system of the raster to, well, lat/lon -- and if the conversion from your coordinate system requires a grid file, like rk says, that grid file should be provided and the conversion through the grid file specified explicitly. I checked and our set of grids (GRIDS.DAT) does not include HS2TN15_NTV2.GSB. It might be that the coordinate system info is fine and it is the checks against known lat/lon that are wrong (although I completely agree it is worth setting up coordinate system info so that the checks work, too).

I don't believe this is HS2 survey grid. I don't believe that HS2 Snake Grid is defined in any standards as it is proprietary and you need a licence to use it. QGIS, ArcGIS don't understand it either.

It's likely that British National Grid (EPSG:27700) is used as a proxy, so all project-internal coordinates will match up, but the coordinates will be in the wrong real-world place a few 100km somewhere else (somewhere in mid-Wales usually!)

What you might need to do is check that it's interpreted correctly as EPSG:27700 (assuming that is what Propeller Aero put it in, as I also doubt it understands Snake Grid)

I sometimes find that Manifold misinterprets the coordinate system, but changing it manually to EPSG:27700 fixes the issue. But maybe I should give more details the next time it happens to me, as sure there could be a fix.

I sometimes find that Manifold misinterprets the coordinate system, but changing it manually to EPSG:27700 fixes the issue. But maybe I should give more details the next time it happens to me, as sure there could be a fix.

(Speaking from memory ~1.5years back)

I think you are referring to a common case when M9 reads the coord sys of some datasource not by epsg code but by parameters. Even if all the params match 27700, M9 won't let you use grid 7710 for transformation. You have to reassign EPSG:27700 to your data to get 7710.

And in map window using 27700 data together with e.g. lat/lon data, corresponding items can be few meters off because no grid is used for on the fly transformation. IIRC.

Incorrect. HS2OSTN15 is a fully fledged and recognised UK survey grid which was developed in conjunction with OS. It does not need a licence and is fully supported by Autodesk, Trimble Business center etc?

It is completely different to OS National Grid. If I was to plot our project in OSTN15, it would end up near Pwllheli in North Wales?

FYI Propeller fully understands HS2 Snakegrid. They use the HS2TN15 transformation as a base then cross check with our GCP values and use a site specific calibration file (if needed) to fine tune the image to compensate for any small local errors and give us the accuracy we need from the point cloud. This is then offered as a local grid export from the platform.

As stated above I've tried importing in EPSG:9300 and the problem is still there.

OK, yes you're right, I am a bit out of date.

But the grid shift is certainly not public - no idea what the licencing is, so I doubt Manifold are going to install it as a default grid transformation.

So it comes down to whether you can translate the grid shift yourself into something manifold understands, or whether they can provide a tool for a user to use or import a gsb or similar, as per QGIS, ArcGIS etc. I expect, but don't understand the internals enough.

Would be useful to know the approach taken with grid transformations, they are getting large these days, and surely cannot be included in a 50MB install, extremely convenient thought that is.

We already have all necessary machinery - we recognize the transform from EPSG:9299 / EPSG:9300 (HS2-IRF, this is what is needed, right?) to EPSG:4258 (ETRS89, lat/lon) that uses a grid, that transform is part of EPSG. It is just that we don't have the grid file (HS2TN15_NTV2.GSB). Probably because the grid file is not public, yes. But if someone already has the grid file (licensed under terms that allow its use in a local install of Manifold), we can use it - it just has to be put into ~/shared, eg, near GRIDS.DAT, so that we can find it. Then one can take a component in EPSG:9299 / EPSG:9300 and convert it to EPSG:4258 specifying the NTV2 transform and the conversion will use the data in the provided grid file. (Or the same could be done via SQL.)

Manifold 9, latest version. I don't think that I could resize the image whilst maintaining the coord system. Just checked Propeller for one of our first flights done a year ago which was only a part of the site and this is 5GB. Registration is being checked to a series of GCPs (Ground Control Points) scattered all over the site which are accurate to sub 1cm. Can a user edit the grid file?

I don't think that I could resize the image whilst maintaining the coord system.

Sure you can. See [this topic](https://manifold.net/doc/mfd9/index.htm#example__resize_an_image_using_merge.htm) and [this topic](https://manifold.net/doc/mfd9/index.htm#example__change_the_pixel_size_of_a_terrain_elevation_image.htm) for two different ways to do that.

Note that, by definition, resizing an image necessarily changes the coordinate system because the size of the pixels is part of the full coordinate system definition. But I suppose you mean in the sense of higher level coordinate system characteristics, such as lat/lon vs. orthographic, etc.

Hi Dimitri, I can't see the logic in resizing an image that has not imported correctly in the first place? The problem is that Manifold is not importing the TIFF into the correct coordinate system to start and therefore I would presume that any edits/exports after will maintain this error? I stand to be corrected though?

it's trivial to resize a geotiff or equivalent, it's guaranteed to have the exact same extent afterwards - if it doesn't then you (and we!) have MUCH bigger problems

*extent* is the xmin, xmax, ymin, ymax of the outside corners, usually (but it depends) this is stored as a *transform*, i.e. an offset value, and scale (width and height of each pixel) - when you resize, the scale changes but the offset does not, so the implied extent stays the same

yes there are half-pixel offset conventions and there is more to the implicit conversions and different ways formats store extent ... etc. but none of that matters for a resize (if things are working at a basic level :)

all this means that if there's a lining-up problem, the resize is so trivial as to be practically guaranteed that the lower resolution version will show the same misalignment (and so I couldn't help chiming in).

https://github.com/mdsumner

I can't see the logic in resizing an image that has not imported correctly in the first place?

I didn't intend to comment on the import issue you had, just the sentence about not knowing that an image could be resized while maintaining the coordinate system.

Reading through this thread it could be you meant adamw's comment, but in his post I believe he meant to ask if you could use whatever software you used to create the problem GeoTIFF to create a smaller GeoTIFF that exhibited the same problem on import into Manifold.

Regarding your import: that's not something I could help debug without having the original file that you're trying to import. With the file in hand, then it's easy to import the file to see what happens, and to examine the file using various tools to see what it claims about itself.

Regarding:

I can't seem to find an option to just use the projection as provided in the GEOTIFF.

That's because Manifold automatically reads the projection info that's in the GeoTIFF. No need for an extra option to specify that. See the GeoTIFF topic.

If Manifold is not importing the GeoTIFF into the correct place, the first thing to do is to stop any discussion about exporting to ECW or anything else. Focus on getting the image imported with all coordinate system parameters set correctly. Talk about anything else just confuses things.

The second thing to do is to get a copy of the file that is being imported into the wrong place into the hands of the people who are attempting to find the problem. If you can't use your original software to generate a smaller file that exhibits the same problems, well, then find a place from which people can download the 60 GB file you have.

There are possibly four things going on: a) something in the projection information in the GeoTIFF tags is incorrect. (very common), or b) Manifold is incorrectly reading or parsing or otherwise constructing the correct coordinate system from the tags in the GeoTIFF (rare, but stuff happens), or c) both a) and b) are going on, or d) something else (extremely rare) is the problem.

Figuring out which of the above four is going on starts with having the file in hand. Without that, it's pretty much just random guesses and very generic advice.

As for generic advice, if you know the coordinate system the image is supposed to use, as in your post that begins with

Found this on another forum for importing into QGIS if any help?

Well, just use Repair Initial Coordinate System to specify that coordinate system (see the doc for how to use that dialog). If that is, indeed, the correct coordinate system, the image should line up correctly. You're simply using Transverse Mercator with a specified datum. Because it's an image there may be some local scale parameters to set, etc., but the basic projection seems to be straightforward.

Hi Dimitri and thanks for your reply.

For some reason yesterday, when I was answering some comments, some of my replies did end up against the comments of the wrong poster. Not sure if it was the forum playing up or just me as it was a rather busy day.

Regarding your suggestions, I confirm that the smallest image that I could supply would be a partial one of the site and would be about 5GB. I'm not sure as to how I could get this to anyone but I am open to suggestions if anyone would like to take a look?

With regards to the possible cause of the problem, I would not say that it is a) as both Global Mapper and QGIS are importing the file with the correct projection. I would therefore think that it may be an issue either with Manifold not liking custom projections on import or it is reading it to be a different projection to what it actually is?

With regards to using Repair Initial Coordinate System, this is something that I will try when I get some time? Thanks

If the file is only 5GB, there are file sharing sites out there you could put it on.

Regarding:

a) as both Global Mapper and QGIS are importing the file with the correct projection.

Maybe yes, maybe no. Tech support never takes anything for granted, so I follow their example.

For example, there are plenty of cases where people say "I'm importing this file..." but they don't say that there are also some auxiliary files with that file. There are software packages which will read what's in the auxiliary files and ignore what's in the GeoTIFF, so you can end up with a situation where the GeoTIFF is wrong and if Manifold correctly uses the GeoTIFF the import is off. But then some other package that gives the auxiliary files higher priority (which is wrong according to the GeoTIFF spec) imports it right.

There are also cases where other software reads something incorrectly that Manifold gets right, so in the case of a double-error where the GeoTIFF is off but the other software also makes an offsetting mistake the result could be right, when if the GeoTIFF spec is honored the result should be off. The classic example of that is a file that announces an EPSG code that requires flipped coordinates even though the data is not flipped, but other software ignores the requirement to flip coordinates, so even though the EPSG code is wrongly announced the flawed software gets the import right. If Manifold uses the EPSG code as the EPSG standard requires, it gets the import wrong. (The error there is not Manifold's, it's an incorrect EPSG code that was assigned.)

I'm not saying that's what's going on in your case, I'm just giving the above as examples where it's a mistake to take anything for granted. It all starts with having the file in hand, and then you can be precise about finding out what's going on and fixing whatever the problem may be.

It's true that sometimes guesses without having the file in hand can find a solution, but it's usually a whole lot quicker to just get the file and start with that. If you want to report a bug or get tech support to help, you'll have to provide it in any event, so may as well put it online so it can be downloaded.

Hi Adam,

I have HS2TN15_NTv2.gsb. Would you know where in the program I need to drop or add this please? Thanks.

Add .gsb to ~\shared folder.

https://manifold.net/doc/mfd9/reproject_component.htm

In addition to grids known to the EPSG system that are published within the grids.datfile, there are grids with such niche constituencies that they do not appear even in the exhaustive EPSG database. Occasionally new grids pop up as local governments publish new grids. When grid files are published in standard formats like .LAS/ .LOSfor NADCON and .GSBfor NTv2, Manifold can use those when they are placed in the~\sharedfolder of the Manifold installation.

Steve, let's take it from the start.

but when I import into Manifold I am getting a shift/rotate and small scale error.

How do you determine that you have shift error? rotate error? scale error?

You must compare the imported image with some other data. What data? How do you compare?

When I import to Manifold though, I can't seem to find an option to just use the projection as provided in the GEOTIFF, as can be found in Global Mapper etc?

We do not know what is provided in the GEOTIFF? Please, paste gdalinfo output or srceenshot from QGIS or GlobalMapper showing projection info.

Also paste what does Manifold say about the projection after import. Use Copy

Attachments:
copy_projection.png

Hi rk and thanks for your reply. I did try to answer yesterday but I think due to the amount of images I attached to explain your questions, my post must have bounced? Therefore I'll try to answer this time only with less images.

Regarding confirmation of the shift, rotate and scale error, we have a series of GCPs (Ground Control Points) around the project which are basically optical targets, which can be clearly seen in the image. These GCPs are coordinated to sub 1cm accuracy in HS2TN15. By comparing the true coordinates of any one of these to the one given by Manifold in the imported image we can compute the shift at this point. By using any 2 of them near to the opposite extremeties of the image, i.e. the longest baseline, we can compute scale and rotation etc.

When I checked the projection info given by manifold after import it is saying EPSG:9300. However in saying this, I did change the default settings prior to the last import attempt, so that all imports should be to this transformation? I can't remember what the default setting was, in to try this again but I can tell that it is making no difference as the imported image is in the exact same position. I have confirmed this from some notes that I made on the given Manifold coordinates for some GCPs from the first import.

Regarding the same projection info from QGIS (image confirmed to be in the correct position), I am getting the following:

WKT

BOUNDCRS[

SOURCECRS[

PROJCRS["unknown",

BASEGEOGCRS["unknown",

DATUM["unknown",

ELLIPSOID["unknown",6378137,298.257222932958,

LENGTHUNIT["metre",1,

ID["EPSG",9001]]]],

PRIMEM["Greenwich",0,

ANGLEUNIT["degree",0.0174532925199433,

ID["EPSG",9122]]]],

CONVERSION["unnamed",

METHOD["Transverse Mercator",

ID["EPSG",9807]],

PARAMETER["Latitude of natural origin",51.9304992,

ANGLEUNIT["degree",0.0174532925199433],

ID["EPSG",8801]],

PARAMETER["Longitude of natural origin",-1.0175018,

ANGLEUNIT["degree",0.0174532925199433],

ID["EPSG",8802]],

PARAMETER["Scale factor at natural origin",1,

SCALEUNIT["unity",1],

ID["EPSG",8805]],

PARAMETER["False easting",231211.675,

LENGTHUNIT["metre",1],

ID["EPSG",8806]],

PARAMETER["False northing",333385.634,

LENGTHUNIT["metre",1],

ID["EPSG",8807]]],

CS[Cartesian,2],

AXIS["easting",east,

ORDER[1],

LENGTHUNIT["metre",1,

ID["EPSG",9001]]],

AXIS["northing",north,

ORDER[2],

LENGTHUNIT["metre",1,

ID["EPSG",9001]]]]],

TARGETCRS[

GEOGCRS["WGS 84",

DATUM["World Geodetic System 1984",

ELLIPSOID["WGS 84",6378137,298.257223563,

LENGTHUNIT["metre",1]]],

PRIMEM["Greenwich",0,

ANGLEUNIT["degree",0.0174532925199433]],

CS[ellipsoidal,2],

AXIS["geodetic latitude (Lat)",north,

ORDER[1],

ANGLEUNIT["degree",0.0174532925199433]],

AXIS["geodetic longitude (Lon)",east,

ORDER[2],

ANGLEUNIT["degree",0.0174532925199433]],

ID["EPSG",4326]]],

ABRIDGEDTRANSFORMATION["Transformation from unknown to WGS84",

METHOD["Position Vector transformation (geog2D domain)",

ID["EPSG",9606]],

PARAMETER["X-axis translation",0,

ID["EPSG",8605]],

PARAMETER["Y-axis translation",0,

ID["EPSG",8606]],

PARAMETER["Z-axis translation",0,

ID["EPSG",8607]],

PARAMETER["X-axis rotation",0,

ID["EPSG",8608]],

PARAMETER["Y-axis rotation",0,

ID["EPSG",8609]],

PARAMETER["Z-axis rotation",0,

ID["EPSG",8610]],

PARAMETER["Scale difference",1,

ID["EPSG",8611]]]]

Proj4

+proj=tmerc +lat_0=51.9304992 +lon_0=-1.0175018 +k=1 +x_0=231211.675 +y_0=333385.634 +a=6378137 +rf=298.257222932958 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs

Extent

Extent not known

Hope this helps?

Sorry for a late reply.

When I checked the projection info given by manifold after import it is saying EPSG:9300. However in saying this, I did change the default settings prior to the last import attempt, so that all imports should be to this transformation?

EPSG:9300 might be fine. The default coordinate system in the Options dialog though - if you are talking about that - is only used for new components created interactively, not during import. But, as I say, the problem might not be in EPSG:9300, but rather in the conversion to lat/lon that is used to check the locations of the GCPs.

To that end, do this: put HS2TN15_NTV2.GSB into ~\shared and restart Manifold. Create a new drawing, set its coordinate system to EPSG:9300 (right in the New Drawing dialog). Open the image, drop the drawing into the window with the image, place the control points. Then convert the drawing to EPSG:4258 and during the conversion, specify that you want to use the conversion with the grid file (see the screenshot, click the circled out button in the Reproject Component dialog, then change the conversion in the Conversion dialog from '(default)' to the NTV2 one):

Then check the coordinates of the control points (Alt-click then check the values in the Coordinates tab in the Info pane).

If the coordinates of the converted control points are what they should be, there is no issue, EPSG:9300 on the raster is correct.

Attachments:
repoject-ntv2.png