Subscribe to this thread
Home - General / All posts - Is there a utility to download and save tiles from a WMS server in the desired projection and use the tiles in Manifold?
tonyw
736 post(s)
#18-Apr-14 06:49

Hi, I'm still trying to find a way to use WMS tiles from our provincial government servers in BC Albers projection. Manifold uses the first projection offered which on this WMS server is Lat and Lon which is not the projection I want. Today I pursued trying to find a program or utility to download tiles in my desired projection and then import or link to those raster images (hopefully they are georegistered). No luck here. Most of the options I could find offered to take an existing map and serve it up as WMS tiles not download them. Other options were technically beyond me. I'm just a user and have no abilities with CGI, PHP or Java. There were some commercial options but the software costs more than Manifold. I previously tried saving the xml file from the Get Capabilities call and modifying it but couldn't successfully connect to the provincial WMS server.

Is there a means of saving WMS tiles? I'm thinking of a program where I enter the WMS url, I choose from among the offered projections, indicate my area of interest, and the program downloads and saves the georeferenced tiles? I can use an open source GIS that gets the correct projection from the WMS server but it's a hassle creating maps in 2 different programs and the final products from two different GIS programs create different looking legends, north arrows, etc.

Thanks

adamw


10,447 post(s)
#18-Apr-14 08:39

You can download the tiles for a WMS image using the Image - Download tool.

I don't think there's an easy way to force Manifold to use the projection you want for a specific WMS image. The easiest I can think of is to create an intermediate WMS server which would generally transfer all data requests to the real WMS server, but hijack the handshake (primarily, GetCapabilities) and reduce the list of available projections to the one you want to force. That's not terribly difficult, but it does require some coding and I am not aware of any canned solutions here.

adamw


10,447 post(s)
#18-Apr-14 08:46

Regarding creating an intermediate WMS server, maybe GeoServer can be of help - see this page in their docs: External Web Map Server.

cartomatic

905 post(s)
#18-Apr-14 08:47

i does the trick indeed


maps made easy - www.cartomatic.pl || www.cartoninjas.net

tonyw
736 post(s)
#20-Apr-14 07:03

Thanks Adamw, Klaus, and cartomatic. I'll check out the intermediate web server, I had stumbled across that but discounted it, not understanding it's potential.. I had also previously tried Klaus's suggestion of modifying the GetCapabilities file and hosting it on my web hosting site, I spent a couple of days on it but didn't succeed in getting beyond the intermediate web host. The getcapabilities file returned from the provincial WMS servers is very long offering many layers and it takes hours to edit out the unwanted projections. I'll try geoserver. Thanks.

KlausDE

6,410 post(s)
#18-Apr-14 11:35

With your own web server you can use a modified GetCapabilities document only and get tiles from the original WMS server. This thread may help. You have to run a web server but no coding is required except editing the original GetCapabilities document. But there are versions of WMS that deliver tiles in unexpected order.


Do you really want to ruin economy only to save the planet?

cartomatic

905 post(s)
#18-Apr-14 13:53

unexpected order - that could be the 1.3.0 specs that obeys the coordsys axis order in the bbox request param. so for some coordsys it will be xy and for others yx.

if the server you are after supports a projection you are after and manifold also does so, klaus's advice should be the most straight forward one. If not, GeoServer should do the trick with wms cascading. Installation is easy, and the docs are good (you will have to limit the number of output coordsys).

if i remember well global mapper has some wms download capabilities.


maps made easy - www.cartomatic.pl || www.cartoninjas.net

tonyw
736 post(s)
#20-Apr-14 19:49

I've been reading about Geoserver. I have to wait until I'm back with highspeed Internet to download and try it. My understanding is I copy the URL for a dataset from the provincial WMS site, add it as a new store in Geoserver. Then in Geoserver I configure/publish the layer(s). I'm hoping I can publish the layer and offer it in only one projection, BC Albers/ESPG 3005 for Manifold. Manifold uses the first projection and I want that one to be ESPG 3005. In reading the user manual, I'm not sure Geoserver caches the tiles locally or just passes them through. I'm hoping it just passes them through.

I did try editing the GetCapabilities .xml file but MS-XML editor is terribly slow and the file terribly big to delete all CRS except ESPG 3005. I never did complete editing one file. There are about 20 such files to edit and if the province updates the GetCapabilitiies file, I would have to edit them all again. I tried search and replace in a plain text editor and that worked more efficiently but the resulting file wouldn't connect to the WMS server.

tonyw
736 post(s)
#21-Apr-14 07:02

I couldn't wait, downloaded Geoserver and JRE. I spent the day experimenting with Geoserver on my laptop. Main steps I followed for cascading WMS in Geoserver:

- Added workspace

- Added store into which I added the WMS URL, this one:

http://openmaps.gov.bc.ca/mapserver/base3?service=wms&request=getcapabilities&version=1.1.1

- Added a new layer from one of the layers offered on the WMS (I chose contours 1:20,000)

- Used Demo to generate the syntax for the new WMS url which I used in Manifold to link an image.

First CRS on the provincial WMS is lat/long which is my problem.

Outcome: The layer even when cascaded through Geoserver is still in lat/long and not ESPG:3005 (BC Albers) as I need. In Geoserver, I left Native SRS = ESPG:3587 then tried variations of setting Declared SRS to ESPG:3587 and alternatively Declared SRS as ESPG:3005 with SRS Handling to "reproject native to declared". No way I could get the cascaded layer to show up in BC Albers, it always came into Manifold in lat/long.

Here's the new URL I used in Manifold generated by Demo Requests in Geoserver

http://localhost:8080/geoserver/wms?request=getCapabilities in File > Link > Image > OGC WMS data

This looks promising, any tips? Here's where I'm getting the provincial WMS URLs

http://www.data.gov.bc.ca/dbc/geographic/connect/index.page

Would love to solve this.

adamw


10,447 post(s)
#21-Apr-14 07:07

http://localhost:8080/geoserver/wms?request=getCapabilities in File > Link > Image > OGC WMS data

Could you attach the XML returned by the above command to your post?

adamw


10,447 post(s)
#21-Apr-14 07:35

I might have some bad news. I checked the EPSG code you provided above - 3005, and it looks like Manifold 8 might not support it. It does support 26907 / 26908 / 26909 / 26910 / 26911, for example, so you might try using one of these (I know none of these are Albers, that's just to see if it works).

I have filed a request to support 3005 and some other codes.

tonyw
736 post(s)
#21-Apr-14 19:20

Thanks Adam,

That was the clue. I made a MAJOR personal break through this morning. Everyone, see the attached JPEG. It may not seem significant but there it is, my test polygon set against a basemap of 1:20,000 contours and water course from the BC provincial WMS server. I've been working on a solution for a long, long time. My earliest question in these forums on WMS projections dates from July 2009, so a five year quest.

Adam, your clue was ESPG 26907 which I looked up to be UTM zone 7N. My area of work is in UTM zones 9N and 10N so I tested with UTM 9N (ESPG 26909). UTM is another accepted provincial projection and is just fine for my map products. Manifold will reproject layers I have in BC Albers (ESPG 3005). The resulting maps are in the proportions and shapes my clients recognize (versus the elongated results east to west when my map products are in lats and long).

I'll write this up more thoroughly when I'm back. Generally the approach is based on Adam's suggestion of using an intermediate server, Geoserver in this case, to pass through the WMS request plus using a projection that Manifold supports. In passing through, Geoserver can reproject the resulting WMS image (excuse the not-techie terminology) to the desired projection (one that is supported by Manifold). I found this document (link at bottom) which describes the steps in Geoserver to link to the external WMS and offer it locally to Manifold.

Once the layer(s) are setup in Geoserver, use the Demos link at the bottom of the Geoserver menu, on the left side, and generate the syntax for the GetCapabilities call. Use the url in Manifold in File>Link Image>Type OGC WMS data.

In Manifold, as others have suggested, for better resolution of the WMS tiles, set the size to 4096000 X 4096000 and tile size to 1024 X 1024.

Other comments

- Geoserver didn't install initially for me using manual start (you'll see this as the default setting upon installation), however installing as a service worked. I guess I'll have it running in the background in windows now.

- Response time. I'm on a very slow Internet connection (1 hour to download the 59 MB Geoserver installer) yet the response from the provincial WMS server is surprisingly quick, 30 seconds to render. Rendering speed should be acceptable on a high speed connection.

Issues.

My next issue is I get a black background to the WMS images. When linking to the image in Manifold File>Link to Image>Type OGC WMS data, set background colour to white. However, this makes for a solid white background so the WMS layer necessarily needs to be the bottom layer or it obscures any other layer. In Geoserver, in Layers>Publish, Opaque is disabled by default. I tested with Opaque enabled but no change. I also set layers, image type in Geoserver to image/png only thinking that would give transparent background. Any thoughts?

Link to instruction document (I can't figure out how to make links)

www.onterrasystems.com/storage/mapsavvy/Geoserver as Admin and setup a WMS Cascade.pdf

Wow!

Attachments:
Eureka moment. WMS and Manifold via Geoserver.JPG

tonyw
736 post(s)
#22-Apr-14 03:49

Can't leave this alone, the vault is finally open. I now have access to the same base geographic and natural resources layers for maps in UTM9/10, a provincially standard projection, as people using Arc__ and Q___. The playing field is levelled. Sharing another test map...

Attachments:
The vault is open (WMS).JPG

tonyw
736 post(s)
#21-Apr-14 21:24

Thanks Adam for submitting ESPG 3005 for inclusion into supported projections.

tonyw
736 post(s)
#29-Apr-14 21:59

Hi Folks,

I haven't had time to write-up how to use Geoserver as an intermediate step for getting WMS layers into Manifold in the desired projection. Checking if anyone would find a write-up useful? Or is the description above sufficient for people to proceed on their own?

I'm back to high speed Internet and the time to render WMS layers through Geoserver is quick and quite usable.

I still have the problem that the WMS layers are opaque so I can't for instance have a WMS layer of contours over top another layer. The spaces between the contour lines are opaque so block out the layer below. If I connect directly to the originating WMS server (not Geoserver) the contour lines are drawn as lines and the space between lines is transparent. I have tried setting in Geoserver for the images to be image/png only but that doesn't help. The work around is to select multiple layers when connecting to Geoserver and the one layer that is delivered has all the desired layers mashed together and I use that as my bottom layer in the layers pane.

The ability to mash together layers using Geoserver as an intermediary has possibilities for bringing together layers from different WMS servers as one layer in Manifold. In Geoserver, you add WMS sources to a Geoserver "store" and all the layers in that particular store are available when in Manifold you connect to that store. So the store can show WMS layers from different sources and mash them together into a common projection. Geoserver can reproject on the fly before sending layers onto Manifold. Thanks Adam for suggesting Geoserver.

jkelly


1,234 post(s)
#30-Apr-14 00:59

See below for an example that I've pulled out of our online maps that use geoserver.

http://xxxxxxx/geoserver/wms?SERVICE=WMS&REQUEST=GetMap&VERSION=1.1.1&LAYERS=your_layer_name&STYLES=&FORMAT=image%2Fpng&TRANSPARENT=true&HEIGHT=256&WIDTH=256&TILED=false&SRS=EPSG%3A3857&BBOX=15978396.392733244,-4575214.765037509,15979619.385185806,-4573991.772584949

I don't use WMS with Manifold, but if you can get that transparent=true into your url that is being called from Manifold, then you should get transparent layers. I can't find how to force this in Geoserver, but I would imagine somehow it would be possible.


James Kelly

http://www.locationsolve.com

adamw


10,447 post(s)
#30-Apr-14 08:56

Yes, you can, by leaving the "Background color" option in the dialog brought by File - Link - Image, OGC WMS Servers (), unchecked.

tonyw
736 post(s)
#02-May-14 07:15

Thanks jkelly and Adam

I tried adding the &Transparent = true clause to the GetCapabilities statement (I don't see the GetMap statement happens in Manifold or Geoserver. I made sure background colour option is off in File > Link > Image. No luck, see attached jpg where the background is still opaque and black. If I move my area down to below the contours, the area is no longer visible. I'll keep working at it.

Attachments:
wms black or opaque background.JPG

adamw


10,447 post(s)
#21-Apr-14 07:04

On editing XML files - just use Notepad. Or Visual Studio, if you have it (a free Express edition will do). Also, instead of deleting data from XML, comment it out (XML comments look like this: <!-- ... -->), that way, if you delete too much, it's easy to fix the issue.

geouser1 post(s)
#09-Feb-16 16:56

Just in case anyone come across the same problem (i.e. downloading georeferenced WMS tiles), I encourage all to have a look at WMS Downloader (http://geotrix.eu/) software. It is available as the ArcGIS Addin as well as a standalone Windows app. Disclaimer: I'm one of the coauthors of this add-in and this is a licensed product. Feel free to check the trial version.

dchall8
1,008 post(s)
#09-Feb-16 17:30

Since this topic has been revived, what is the difference between downloading and caching the imagery? When I had an ultra high speed connection I unticked the Cache box, but now on what amounts to modem speeds, I use the cache for quicker rendering.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.