Subscribe to this thread
Home - General / All posts - New video: Rendering Shootout - Arc / Manifold / Q
Dimitri


6,512 post(s)
#09-Feb-21 14:12

There's a new video comparing rendering performance between Release 9, ArcGIS Pro, and QGIS.

Short description:

See how Manifold Release 9 compares to ArcGIS Pro and to QGIS when rendering 44 million roads. We pop open exactly the same data in all three packages and see who wins.

The shootout features 49 layers that show roads in each "lower 48" US state plus the District of Columbia, 22 million roads in all. To compare rendering of 49 separate layers with one big layer, we also use a single big layer that has all 22 million roads in the same layer. The single big layer is stored in the Manifold project file, in a GPKG database file in QGIS, and in an ESRI file geodatabase in ArcGIS Pro, using the "native" storage for each of the three packages. Guess who wins?

RonHendrickson
274 post(s)
#09-Feb-21 16:22

Really a dramatic visualization of Manifold Release 9 speed versus its main competitors. Although I am now retired, I can envision a young GIS practitioner with Manifold 8 and 9, some SQL and maybe a couple of other language skills being able to make quite a living as a consultant, if he could market himself properly and have some imagination on solving his clients' problems.

dchall8
848 post(s)
#09-Feb-21 23:58

I'm retired, too, so what I did with M8 and no additional languages made me some money simply because of the ESRI mind set among government workers. The local heath department had a GIS/ESRI guy, but he didn't know how to use it. I made a case for hiring me as a consultant using the Dr John Snow information about the 1861 cholera outbreak in one neighborhood of London. I mapped the data several different ways and started with them right away. I talked to their GIS guy who said he could have done that, "but I haven't been trained on that." I got a listing of available data and asked for all of it. I sent them what I considered very basic information and it was like a miracle to them. They had no idea where their restaurants, WIX centers, libraries, walking parks, grocery stores, mini-marts, etc. were relative to neighborhoods, bus routes, bike lanes, roads, school districts, and voting precincts. I gave them a huge amount of information in layered PDF form. About a week later one of the news headlines announced the mayor's initiative to get at least one grocery store in a school district which had only gas stations providing over priced groceries to the families there. He had to have gotten that information from what I provided.

The problem with all the ESRI marketing is they end up exhausting their customers by providing piecemeal training with the promise that buying more training and software add-ons could provide additional benefits. Manifold had it all built in, and Art Lembo provided all the training needed to get going for well under $100. Small government offices generally don't have the vision to see the benefits of the training. Even in the oil and gas business I ran into very well paid ArcView, users with unlimited funding, who were unable or unwilling to draw tens of thousands of numbered, small acre parcels. Working out the kinks took 2.5 days and making the final product took half a day. Not being an ArcView user I don't get their problem.

M9 is certainly fast, but it's not entirely ready for what I needed on those projects. Making the maps is easy enough, but the final reporting is what the user wants.

StanNWT
187 post(s)
#09-Feb-21 20:09

Would you be interested in a 15 million polyline segment river network drawing layer with 318 million vertices point drawing layer, which is a point for each vertex in the river drawing to demonstrate the speed of Manifold 9 vs. ArcGIS Pro vs. QGIS? I have a 158 GB Manifold Project file for all the rivers at 1:50K for Canada. It's one of my early attempts to stress test Manifold 9 back in October 2017. I've got a custom lambert conformal projection for the map which re-projects on the fly the Google Satellite view and Google streets only image services. The river network polylines and vertices aren't broken down into dozens of separate drawings by provinces or smaller segments of provinces to make the rendering speed up, but I'm sure you could do that with a join to the province polygons, or even the NTS map sheet indexes, which I can put into the (.map) file. Just a thought to try and illustrate a significantly larger data set that is essentially based on free publicly downloadable data from Natural Resources Canada's 1:50K CanVec data. I could provide the river network in the original file geodatabase as well.

Dimitri


6,512 post(s)
#10-Feb-21 13:05

I'd love to get the 158 GB project. You could zip it using 7zip (seems to have the smallest size) into a .7z file and upload it to the tech support FTP site if you have a login for that. If not contact tech support and they'll get that set up for you.

For big projects I basically use OSM data. The whole planet PBF is over a terabyte, but that's unwieldy and slow to use. But you can get extractions that are really big.

StanNWT
187 post(s)
#25-Feb-21 15:38

Hi Dimitri,

I have a dropbox account that I can upload the data set to and can send a share link to you, or whomever needs it.

Dimitri


6,512 post(s)
#26-Feb-21 07:30

That would be super! I'd just send it to tech support, so if you email them with a link to the dropbox data they'll get it. Thanks!

StanNWT
187 post(s)
#07-Apr-21 20:02

Hey Dimitri,

I know it's been awhile, I had lots of other things on my plate, but I've sent three drop box links to tech support for you to download from my dropbox account. Three (.mbx) files.

1) The 141GB 30m DEM from the USGS NED -> now 3DEP data set. (89W - 180W, 57N - 85N). I have 10m contours generated in Manifold of this data set as well. I haven't gotten around to attempting the 1m contour interval with that data set as of yet.

2) NRCan CanVec 1:50k rivers for Canada with the decompose to coordinate points of that data set. 15 million rivers and 318 million decomposed coordinate points with all the data.

3) NRCan CanVec 1:50k full database for all point, line and polygon features as well as the accessory tables that can allow for a more feature rich attribution once the relational joins are made.

The CanVec data is from version 1.1.3 circa 2013, but it's was freely downloadable from geogratis in 2014.

Just giving you some really sizable data sets to play around with in a large geographic extent if you want to test out Manifold's performance drawing layers, performing analysis on a big raster data set or vector data sets.

The CanVec 1:50K rivers decomposed to points is a data set that crashed Manifold back in 2018, but was easily fixed with a patch increasing the size of data sets it could work with as well as the RAM memory footprint I believe. Adam and yourself provided a lot of feedback on that data set.

On another note I was able to merge 10GB of LiDAR data in 42 seconds and contour it at 1m interval in 119 seconds, fast enough to do a live demo via GoToMeeting a week or so ago.

artlembo


3,156 post(s)
#09-Feb-21 20:58

that's a great dataset. Do you have it on an FTP site somewhere that would allow downloading of the .map file? I'd like to try some other tests in our lab. Mostly, I'd like to see some attribute and spatial queries.

Dimitri


6,512 post(s)
#10-Feb-21 13:00

Wasn't sure if the request was for StanNWT or me, but the .map project is on the tech support FTP site as a 6.6 GB .zip file called eg gdt all roads.zip Looks like zipping a .map with 7zip is the smallest possible size you can get.

Forgot to mention: for playing around with attributes and spatial queries OpenStreetMap data is great. You can download some really huge PBFs and what's nice about those is that the really complex data structure of OSM seems made for SQL in terms of extracting features that people actually want, like a coastline, that can be pretty hard to detect in the PBF as it is from OSM.

RAR81 post(s)
#10-Feb-21 13:48

Dimitri,

Could post the link please?

Thank you!

Dimitri


6,512 post(s)
#10-Feb-21 14:16

It's on the tech support FTP site. If a video's popular Manifold will post the source data for it in the "downloads" page, so probably in a day or two it will appear.

Dimitri


6,512 post(s)
#10-Feb-21 17:07

The Manifold project is here. That's a 6.8 GB zip file that unzips into a 27 GB .map project.

Dimitri


6,512 post(s)
#10-Feb-21 15:30

Whoops... bad link. Will update when it gets published.

RAR81 post(s)
#10-Feb-21 17:18

OK.

Thank you Dimitri!

artlembo


3,156 post(s)
#10-Feb-21 20:53

yes, I was referring to your project. As interesting as StanNWT's project is, I'm too busy to mess with a 158GB project!

Manifold User Community Use Agreement Copyright (C) 2007-2019 Manifold Software Limited. All rights reserved.