Subscribe to this thread
Home - General / All posts - M9 hardware for working with LiDAR data Revisited
danb

2,064 post(s)
#17-Oct-23 03:36

Another locked thread.

A couple of years ago I received some great advice about a purpose designed Manifold LiDAR workhorse and the following specification was arrived at given Manifold's current and anticipated architecture and plans at that time (M9 hardware for working with LiDAR data (georeference.org))

Case: Phanteks Enthoo Pro Black Full Tower Gaming Case

PSU: EVGA SuperNOVA 1000W G5 80+

CPU: AMD Ryzen 9 5950X

MOBO: ASUS TUF Gaming X570-PLUS (WI-FI) ATX

Cooler: NZXT Kraken X73 All in one Liquid Cooler

RAM: G.SKILL Ripjaws V Series Black 128GB DDR4 Desktop Memory 3600Mhz (4 x 32GB)

HDD1: Samsung 980 Pro 2TB NVMe PCIe 4.0 M.2 SSD

HDD2: Samsung 870 EVO 4TB SSD

GPU: Leadtek Quadro RTX4000

This machine has been truly excellent and between it and Manifold will have paid for itself many times over in saved labour hours.

Anyway, another colleague is now looking to build similar for a similar set of tasks and I thought that I would ask if there were any major changes and updates to the hardware specification suggested back in June 2022?

As always, any thoughts and suggestions would be much appreciated as I am a bit out of my depth with this stuff.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


7,449 post(s)
#17-Oct-23 13:25

Looks like an outstanding machine. I especially like the PCIe 4.0 M.2 SSD. The AMD-based motherboard can run that at top speed.

The only comment I have is that the Quadro RTX4000 seems overpriced. You can get around 5000 CUDA cores for about half of what those go for. Since most of the bang with GPU parallelism happens in the first thousand or so cores, it's usually more bang for the buck to not overspend on GPU but instead to put the more/faster SSD, like two 4TB PCIe 4.0 M.2 SSDs instead of one 2TB.

danb

2,064 post(s)
#17-Oct-23 22:24

Thanks Dimitri, this is very useful. It has been a great computer and that is largely thanks to yours and Tim's input back in 22.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

dchall8
1,012 post(s)
#18-Oct-23 08:28

When I was shopping I looked at building my own, but setting up and testing the firmware got me down. I ended up buying a custom Alienware desktop from Dell. This is similar in terms of today's hardware.

Building a computer is sort of like building a rifle from parts on the Internet. Just because the parts fit together does not mean they will be reliable as a unit.

danb

2,064 post(s)
#18-Oct-23 19:15

Thanks dchall8. Understood re. your last paragraph. This is why I came here originally. I know that Tim and I feel sure that Dimitri have a full grasp of this and that is why we ended up with such a great computer.

In fact this is why I ended up looking at a custom build in the first place. Tim (tjhb) was kind enough on a few occasions to help me with my work box upgrade when we had to buy from Dell. Many of these though looking impressive were often hobbled in some way by a mismatch of componentry or just overtly expensive from the outset.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

danb

2,064 post(s)
#31-Oct-23 22:16

Our IT department has been looking into getting a setup which up to 25 power users will have access to (thought it is unlikely that anything approaching that many would be working on it at any one time). The attached is where they are at at the moment and any commentary positive or negative from those in the know and especially the factory would be most welcome.

I know they are also looking into the cost of the Gold 6448Y which as double the number of cores.

The intended use is for LiDAR, Machine learning, Remote Sensing and other modelling endeavors

Attachments:
Clipboard-1.png


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Dimitri


7,449 post(s)
#01-Nov-23 07:09

I don't know about that... with only 20504 CUDA cores given the two A40 cards, are you sure that will be enough? :-)

OK, more seriously, it's obviously a very intense and powerful configuration. Whether it is the best for what you want depends on the details of what you're doing. For example, two A40 cards are way overkill for GPU computation using CUDA cores. They're also overkill for most rendering tasks, unless you're rendering frames for a CGI-heavy Hollywood movie. Of course, if your users are working with virtual workstations and using the server for rendering using 3D software that works with that configuration, then maybe 25 of them will be able to keep the A40s occupied. It all depends. On the other hand, doing AI/machine learning the tensor cores will be really super.

My initial impression is that two CPUs with 16 cores each seems light on CPU cores (total of 32) for a machine that's intended to be time shared by 25 power users. The new Xeon Gold series are very good technology and are fast at single threads, so nobody's going to get fired for choosing those, but with many users my guess (just a guess) is that you'd do better with more cores where each individual core might not be as fast as an individual Xeon Gold core. Having 32 or 64 slightly slower cores per CPU probably would give you more throughput overall with 25 power users than having 16 slightly faster cores per CPU.

That's where you get into the hall-of-mirrors labyrinth of configuring more complex machines based on the cost of various deals you can put together. If your IT department is buying a ready-built machine from Dell, they're going to be constrained by what Dell offers, as compared to a mix and match setup put together from the open market.

My own, personal experience in this has been that you get the best bang for the buck in terms of manycore processors by leveraging higher quality consumer gear or the lower end of "server" gear, buying not the very latest CPU generation but usually a six month or one year back generation and loading up with lots of cores, RAM, and fast M.2 SSD, going for more cores on the CPU and one CPU instead of two CPUs. That's especially true these days when extra aggressive pricing on a new CPU generation seems to be a thing of the past.

For example, in the past I've had great luck buying AMD Threadrippers one or two generations back that have a gonzo number of cores for an absurdly low price than the latest Intel chip with fewer cores that has a higher price. But there are a lot of sensible reasons why IT departments often resist that approach, given the very high cost of maintaining ad hoc system configurations, and especially if you start out with technology already a year or more old.

In the case of your proposed system, it looks like the architecture is balanced to do the really high speed, intensive computing (massive AI work, etc) on the GPUs with the CPUs playing a support role. I'd therefore go with more cores in the CPUs so there are always extra cores to handle the many small tasks that you get with 25 users. How that works out most optimally in terms of price and comparison between Intel and AMD will take a lot of tinkering with spreadsheets and available deals.

I'll close by emphasizing that the above is just a gut feeling, not a real analysis. The performance you get and whether users are happy or not with big, complicated, expensive configurations will depend very much on the specifics of software being run, from OS to applications, and how the various software packages are used.

danb

2,064 post(s)
#01-Nov-23 22:28

Thank you Dimitri, this is really helpful and very much appreciated.

I think the spec came originally from an ESRI rep (which may have some bearing) and my own feeling was that it was light in some areas and very heavy in others. It is great to have this sense backed up by someone who knows far more about this stuff than I.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

danb

2,064 post(s)
#06-Nov-23 21:38

And one more (from a colleague in a different Region that has also acquired all of region LiDAR) ...

  • NZXT H9 Elite Edition ATX MidTower Gaming Case Tempered Glass, White, 3XA-RGB Fan and Controller Pre-installed, CPU Cooler Support Upto 165mm, GPU Support Upto 435mm, 360mm Rad Supported, 7XPCI Slot, Front I/O: 2X USB, 1XType C, 1XHD Audio.
  • Gigabyte NVIDIA GeForce RTX 4080 Aero OC 16GB GDDR6X Graphics Card 3.5 Slot - 1x 16 Pin Power (3x 8 Pin Power Adapter Included) - Minimum 850W PSU
  • NZXT Vertical GPU Mounting Kit --- White H7, H5, H9, 175mm PCIe 4.0x16 Riser Cable Included
  • AMD Ryzen 9 7900X3D CPU 12 Core / 24 Threads - Max Boost 5.6GHz - 128MB L3 Cache - AM5 Socket - 120W TDP - Integrated Radeon Graphics - Heatsink Not Included
  • NZXT Kraken ELITE 360 360mm AiO Water Cooling with 2.36 inch square LCD Display, for Intel Socket LGA 1700 / 1200 / 115X, AMD AM5 / AM4 / sTRX4* / TR4* (*Threadripper bracket not included)
  • G.SKILL Ripjaws S5 64GB DDR5 Desktop RAM Kit - Black 2x 32GB - 5600Mhz - CL36 - 1.25V - 36-36-36-89 - F5-5600J3636D32GX2-RS5K
  • ASUS TUF Gaming X670E-PLUS (WI-FI) ATX Motherboard For AMD Ryzen 7000 Series CPUs Socket AM5 - AMD X670 Chipset - PCIe 5.0 - 4x M.2 - 3x Internal USB 2.0 Header -1x Internal USB 3.2 Header - 1x Internal Type C Header - 1x Internal TB Header
  • Kingston NV2 4TB M.2 NVMe Internal SSD PCIe Gen 4 - Up to 3500MB/s Read - Up to 2800MB/s Write - Backward Compatible with Gen 3 - 3 Years Warranty
  • Samsung 870 EVO 4TB 2.5" Internal SSD V-NAND - SATA3 6GB/s - Up to 560MB/s Read - Up to 530MB/s Write - 7mm - 5 Years Warranty
  • be quiet Pure Power 12 M 1200W Power Supply 80 Plus Gold , ATX 3.0 PCIe Gen 5 Ready - 12VHPWR 12+4-Pin, 10 years Warranty
  • plus operating system

As always thoughts and comments from those in the know are extremely valuable and go a long way to improving working life on a daily basis.


Landsystems Ltd ... Know your land | www.landsystems.co.nz

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.