After a false start or two I was able to Knock Out Pixels using Join and Alpha. The Mask field, added off camera, tripped me up. That process is not intuitively obvious at all.

At 7:57 into the video, the narrator suggests the approach could be used to view imagery over terrain elevation data. Here's where I am so far on that project.

Here's a detail of GEOTiff topo Image from US Dept of the Interior dated 1964

Next is the same detail at 50% transparency

And finally the same detail with DEM turned on underneath.

Note the loss in detail in the GEOTiff when making it translucent. Others making these types of maps are using Blender to drape the tiff over a terrain created from a GIS program. I'm hoping Manifold can cut out the Blender step to create a full range of contrast in the resulting tiff image. And I'm thinking the solution lies in the Alpha and Join manipulations, but I'm missing some logic to join the 3-dimensionality from the DEM into the GEOTiff. Can that be done in another channel and styled to give the hillshade effect?

Attachments:
64 GEOTiff, transp and contrast w-DEM.jpg
64 GEOTiff, transp.jpg
64 GEOTiff.jpg

This might be an annoying answer, or not an answer at all, but the difficulty here is that we don't have blending modes between layers in Manifold. Or rather, we have just one. I have never wondered deeply enough what it is, but I imagine it is just a simple weighted averaging of pixel values converted to display RGB, something like underlying layer(s) + overlying layer x opacity.

This means that any blending necessarily reduces contrast. (Not detail, but as you say, contrast.)

What we really want is to lay down the colour layer(s)--here the 1964 topo image--then overlay the shading layer and only use it for shading--not to blend it as if it were also a colour layer. (This is the reverse of your layer order, but neither order is definitively right or wrong.)

That is not built in to Manifold.

You can use another product. I don't use Blender but that is just (lack of) habit. I would use either Luminosity, Lighten+Darken, or Overlay blending mode in Photoshop--or more likely a combination, maybe using multiple shading layers from different angles and so on.

We can do any or all of this in Manifold 9, because it is just channel math. We have infinite flexibility. But it is a manual process, and fairly static. Although we can use SQL expressions to blend channels "live", visual trial and error is a bit tricky. (I'm sure we can make transform previews work well with some thought.)

Another technique I have sometimes used to use a shading layer to screen itself, i.e. as its own alpha channel (or the reverse).

If you want to try some things out together, test data would help.

Tim,

I"ll write up the layer blending as a suggestion. Have blending in the transforms would be beyond useful to me. Blending to output quality shaded reliefs is key to some of the work I do. Having now spent time in M9, having access to polishing tools like blending would be an advantage.

Access to Leyland Brown's Terrain Texture Shading functions from within nine would make this a killer application! Leyland has released some open source code for incorporation into other applications. Again, something for me to chase up.

P.s. for interest (link corrected): http://www.georeference.org/forum/t146243#146298.

2007?

Ogres?

An excellent suggestion for layer blending.

(Yes that is a good link too. Nostalia! But I edited and switched it.)

Texture shading is interesting. It's simply an elementary use of fractional Laplacian operations (see Leland Brown's discussion at the end of http://www.mountaincartography.org/activities/workshops/banff_canada/papers/brown.pdf ) that Brown implements using interpolations simplified to convolution matrix operations, which, of course, is very easy to do in 9. It's even amenable to parallelization.

As adamw has remarked, there's work on deck for things like georeferencing, vector editing, and labels, but it's clear there are many "low hanging fruit" benefits from expanding operations with rasters. I've written up a suggestion for texture shading as one of them.

Thanks Dimitri,

at M9's new price point, having a functional terrain texture shader would be a cartographic killer!

I've been using the command line version on an old Mac to do the work. Dealing with DEMs larger than 50Mb is starting to stress the old Mac somewhat.

Somehow my post is taking on a wonderful life of its own. I'm hesitant to add one more picture, but since Tim mentioned contrast, I thought I'd post one with the contrast turned up in Manifold Style. This is the 1.0 stdev setting for the RGB channels.

Whites have taken a pink hue and lines seem to have a slight halo. Not the quality we're all expecting.

Attachments:
64 GEOTiff, transp and very high contrast w-DEM.jpg

Whites have taken a pink hue and lines seem to have a slight halo.

You can adjust that by setting range differently for the different channels, instead of setting the range by choosing the same autocontrast settings for all three channels.

Yes. But there may be a trade-off, I think, between treating image channels as data, and treating them as colour.

Sometimes we want one thing, sometimes the other, and naturally we always expect the "right" answer.

Treating images as data means just numbers and math. (And maybe pink glasses.)

Treating channels as colour means allowing for human perception, and non-uniform scaling, hopefully within a defined colour space.

Solving this is going to be really tricky, since there is not one right answer.

If I had to guess how it might work out in the long run, I would imagine that treating channels as pure data would remain possible (as it must), but maybe it would not be the default for multi-channel images. There might need to be a checkbox to allow treating 3- and 4-channel data as RGB(A), entailing the application of perceptual scaling both for styles and transforms.

(I have noted before that I am moderately red-green colourblind. Probably some other members of the forum are in the same category. I am not entitled to make strong suggestions on this topic.)

My old workflow was to export both the shaded relief, and the image overlay into Photoshop, use layer blending to get the desired outcome. That way I could use M8 to build a shaded relief, a hyspo tint, a terrain texture all at the same resolution, and extents, then export for blending.

I see that QGIS now offers layer blending modes: Normal, Lighten, Screen, Dodge, Addition, Darken, Multiply, Burn, Overlay, Soft light, Hard light, Difference, and Subtract.

Blending in M9 will be a case of understanding the modes, and then being able to write the SQL update to get the desired outcome.

I'm way behind in sending suggestions to sales, or posting my wish list here. Must make amends.

<edit> snap with Tim's response

Yes that was a perfect snap Dale.

http://www.shadedrelief.com/dem/yosemite.mov

Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

I was thinking more like what Scott Reinhard is doing. He's taking the free USGS historical maps, for example Mt Tamalpais, from 1897, and turning it into a 3D vision using Blender. Note that the attention to detail is amazing. The light is from the west so the east side of his image gets the shadow of the terrain.

That's nice work.

You will be able to achieve the same results with layer blending in Manifold (artistic skill notwithstanding)

The preview in M9 may prove to be very useful, as Tim says above. Part of the artistic side is being able to see immediately the effects of your inputs, be that illumination elevation and azimuth, or blending modes.

As an interim approach, try generating a shaded relief in grey scale, so shadows only, no RGB.

Consider a blur of some sort to generalise the shading, defaults give a hyper real look. Then select the white pixels and delete. You want a layer that has only shade, and transparent pixels for the rest. (If that makes sense) Then with a light touch, use layer transparency to add the shading back in.

If you have an interest in historic USGS topo GEOTiff files, ESRI has worked with them to create this website for downloading free images.

http://www.manifold.net/images/flood_ani.gif

I post the link because browser don't support quick time nowodays so best was to download it and this need to go to html source code .

flood_ani.gif is really great !

So manifold 8 ( with surface tool I think) can do that but not Manifold 9 ( I think) ?

Is there ll be a way to automate capture Location area when cell value ( define auto increment value ) or range of selected row are process one by one ?

3D VIEW OFTHE GRAND CANYON FLOODING

Click on the thumbnail to see a 780 KB animated gif of the Grand Canyon flooding in Manifold System. This animated .gif was created using a DEM data set of the Grand Canyon imaged in a terrain window in Manifold, which provides an automatic 3D view of elevation data. Manifold can draw a semi-transparent "waterline" plane through a terrain at a given level for waterline analysis. The flood effect was created by setting the water level higher in ten even steps and then making an animated .gif of the results using Microsoft GIF Animator.

regard's

it is a shame that this tutorial link is dead http://www.manifoldsoftwarelimited.info/doc/create_a_globe_image.htm

Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

it is a shame that this tutorial link is dead

That's a topic in the Release 8 user manual, at

http://www.georeference.org/doc/manifold.htm#create_a_globe_image.htm I've reported the broken link on the page.

An interesting thread.

As was said, we can do everything using raster math in queries / scripts, but that's too laborious and it'd be better to have built-in tools.

To recap, the ideal set of tools would be something like this:

• for easy cases - an option of combining colors from one image with relief from another image without involving opacity, just treat the first image as if it was a texture and apply shading on top of it to avoid reducing contrast,
• for more complex cases - a transform to put shading into a raster of its own so that it can be combined with other rasters, and a set of functions to combine rasters using various blend modes.

Correct?

I'd not considered the functions as easy/complex.

It makes absolute sense to approach this as you have described.

Easy cases would permit a user to merge a simple shaded relief, with a hypsometric tint, a shadows only image with a full colour image, or a terrain texture with a shaded relief. That will produce a far more polished output that the default shaded relief.

Complex cases, a transform to extract shadows would be nice. Patterson T. has a little paper on filtering and DEMS Shadows with alpha channel for transparency would have been fit for purpose with dchalls example above.

A set of blend modes to combine rasters would be very desirable in my instance.

Does it make sense to implement this when, serif affinity Photo do this for 25 $ in microsoft store during covid ? Here forum post about blend mode. I always think software should share data beetween them , Microsoft has COM DCOM and never beat by CORBA ORB RMI ? The way android softwares share data or use tiers software gui is wonderfull !! Does Manifold ll be the new Adobe Firework's software that mix raster vector layer !

If manifold could use blending dll in affinity photo without re invent the wheel it ll be nice. I really like in audio the cubase VST plugin that many raw audio software implement !!

Photoreactor work like that and can create filter that can be use in photoshop !!

https://affinityspotlight.com/article/blend-modes-explained/

https://forums.getpaint.net/topic/16607-blendmodes-plus-v235-32410/

https://github.com/toehead2001/pdn-blendmodes-plus

Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Adobe buy ads when search serif affinity !! all has been said

Attachments:
Screenshot_2020-06-16-23-16-12-159_com.android.chrome.jpg

Book about Science , cosmological model , Interstellar travels

Boyle surface fr ,en

Having these options in Mfd would be a huge plus given the large images Mfd works with compared to photo software.

Another thought is would it be possible/desirable to have a dialogue that allowed blending with 3 layers instead of two? For GIS work, it is nice to combine 1) hillshading from a DEM, 2) texture from an image/aerial, and 3) a land classification vector/image. This can create a beautiful and informative images. Assuming this is readily doable, having a 3 layer dialogue might make the task above easier.

Correct?

In my opinion, no, not really.

I would skip the option "for easy cases" completely, because it overcomplicates things and does not address the issue. I would only do what is necessary (further below), plus fixing...

a transform to put shading into a raster of its own so that it can be combined with other rasters

That is already built in, but it's horrible to use: Style, channel 0, unique values, 1 break, tally, change colour to white, update style--this should all be one click, at most two (or at least a handy default, after all there is Blacks to Whites, why not just plain White? a mystery)--then apply shading. It was just as bad and mysterious in Manifold 8.

Apart from that, what is needed is

a set of functions to combine rasters layers using various blend modes

It should not be a set of functions, but a set of options, available for all map layers, not only for rasters.

Nothing else is needed, but that is essential.

Worth bearing in mind: it is normal to want to apply more than one shading layer to a map, with different angles, different weights or opacities, possibly made from different elevation images (or from curvature, as Dimitri also puts to great effect). I often combine four or more different shadings. Always after much experimentation--it would also be worth making interactive experimentation easy.

I always do this in Photoshop because Manifold has not thought it through.

I always do this in Photoshop because Manifold has not thought it through.

Respectfully disagree. There's a difference between thinking it through and deciding to implement what the thinking through leads to.

Photoshop does nothing but rasters. It's totally useless for vector work, for example. It also does absolutely nothing with attributes or spatial context. And it is absolutely terrible at big rasters. That's OK, because Photoshop aims just at smaller rasters, as encountered in photo work, thinking through all that matters for those, and implementing what the thinking through leads to.

Manifold has also thought through what a full articulation of raster capabilities means. That's why the internal infrastructure is what it is. But Manifold has not yet implemented what all that thinking through leads to. Instead, only the bare minimum of capabilities are there, and at that only those which fit in a lowest common denominator orthogonal set of controls related to channels and ranges, have been implemented.

Raster manipulation is easy stuff with very well known and published approaches to things like shading, so there's no reason why, once it bubbles up in priority to the level of being implemented, there could not be a wide range of point and click (not SQL) controls that automate and otherwise provide conveniences for a wide range of facilities for things like blending, shading, etc.

I expect that naturally will gain priority, as the new, low price of 9 is already starting to transform the user base into a constituency with broader interests in presentation, and point and click raster manipulation as in Photoshop. That's a different set of priorities than the more data-centric, spatial ETL orientation surrounding a hard core, SQL oriented, data engineering tool selling for higher price. You never see FME users talking about blending or shading modes, for example.

There's also the steady determination at Manifold that anything possible in 8 would be available, and better, in 9. I always enjoyed the many raster editing tools in 8, with some things actually easier and better than in Photoshop, so it will be nice to have analogs in 9, but analogs not held back by the limitations in 8. Doing texture shading on the fly in 8 would have been slow, for example, but 9 is perfect for such things.

It would be great to hear what people think should be the priorities, what raster items to add first (so people and system can cogitate on what's best to do next), then next steps and so on.

My top three for rasters:

1.Mouse selections (rectangular, free form etc) by pixel and not only by tile, so one can next crop, copy paste etc the pixels.

2. Georegistration

3. An equivalent to M8 Tools-Make image. (Not just raster this one)

Thank you.

2 (georegistration) is coming and 1 (selecting pixels rather than tile) is going to come as well.

For 3 (make image), do you mean rendering a map with all the styles applied into a static image? Because if you mean transferring vector data to raster for analysis, we already have that in the Join dialog.

For 3 I mean rendering a map (or a drawing/image/wms server image) to a static image.

You are quite right Dimitri. My last sentence (the second part) was neither helpful nor true. I went back to delete it, but was slightly too late.

Mind you, some of your comments are in the same direction.

Photoshop does nothing but rasters. It's totally useless for vector work, for example.

You overstate that. Photoshop can import vector linework (in AI format) as paths, and can also export paths as vectors. This data can also be geographic. (It can even be georegistered, not hard to achieve but not very obvious.) Probably few users know this, and fewer know how to make good use of it, but it is true. Paths can remain vectors within Photoshop, and they can be used for drawing lines (and even gradient fills), but, no surprise, they are mostly useful for manipulating pixels.

On the other hand I would agree, if you would meet me this far, that "vectors in Photoshop are irremediably clunky". No one would really use them by choice. There would always always always be a better tool.

And it is absolutely terrible at big rasters. That's OK, because Photoshop aims just at smaller rasters, as encountered in photo work...

That is false. Photoshop handles extremely large rasters extremely well--provided that they are in its native format (PSD or PSB). (It does this with special handling of TEMP space.) Interoperability, though...

No useful interoperability (at least for us) unless you invest in Avenza's Geographic Imager Photoshop extension. My licence has long since lapsed, though I am often tempted to renew. It is excellent. Manifold should have this extension on its desk, and should replicate everything in it, even if that would be commercially cruel.

Also, Manifold should invest time and money in a PSD/PSB import filter.

Why on Earth can't it already read PSB? Write would be very good too.

Just to be sure, when we are talking about importing PSD, ideally you want not only the final merged image, but also the layers + blending modes read into a map with the map rendering into the merged image similarly to how Photoshop does it, correct? What's the intended use of the import / export - to work interchangeably in Photoshop / Manifold, or to start in Photoshop / finish in Manifold or vice versa? If it's the latter, then perhaps it would be best to start with importing / exporting of just the merged image (because that can be done faster than that plus support for the layer structure inside the file).

I had not thought enough about this before your questions.

I think allowing import of PSD/PSB as if they were flat (so, fully rendered) is enough--and ideally export to the same formats (either of image components or fully rendered layouts and maps).

You could even strictly disallow import of unflattened Photoshop images. That would be fine. The benefit in interoperability would stlll be significant. (In a sense, this is only because there is no better interchange format for very large images.)

Would it ever be worth trying to decode everything in a multi-layered PSD/PSB? All layers, effects, gradients, paths, smart objects, 3D objects, the works? Definitely not. Life is too short, software is fragile, and after all who would pay?

I think workflow interoperability only for flat (flattened) images would be the ideal balance.

I think I remember that the PSD and PSB APIs are written in Pascal. Not so tempting probably.

On the other hand I would agree, if you would meet me this far, that "vectors in Photoshop are irremediably clunky".

Happy to do so! Consider us met halfway. :-)

That is false. Photoshop handles extremely large rasters extremely well--provided that they are in its native format (PSD or PSB). (It does this with special handling of TEMP space.) Interoperability, though...

Consider my remark amended to "as practical matter, Photoshop is absolutely terrible at big rasters." Try opening big GeoTIFFs in Photoshop and it's way too slow.

:) yes

and

you are right for any TIFF format, I agree.

(I will have to restate my remaining very good points tomorrow.)

Tim,

that's a very interesting proposition. blending more than rasters, drawings as well. I'd not remotely considered that. Part of the key is interactive experimentation as you put it. Having that preview returning immediate feedback before committing a process would be highly desirable. Such a function would permit poster dchall to both experiment and learn.

I concur with multiple layers, I routinely use three four or more, and like you in PS. I had the good fortune to pay a visit to the NPS Harpers Ferry Center, and meet Tom Patterson, and watch him at work. At the time his workflow was GIS to Natural Scene Designer, then PS. The step from NSD to PS was a publishing step, ie no geographic data retained. PS was all about blending, blurring for distance (gradient blur) and polish ready for the graphic designer to finalise the job.

As an aside, have you ever used a location specific shade? ie shade one part using a different elevation/azimuth Z value or colour (yellow for peaks) to make a ridge or a peak pop just a bit more?

Dale,

We do seem to be a bit on the same page here, I'm glad.

You made me smile, especially with your comment about Tom Patterson. I am also lucky enough to have met him and talked shop (twice, both times in NZ). He's a master, but does not talk down to anyone. All creative ideas are good.

As an aside, have you ever used a location specific shade? ie shade one part using a different elevation/azimuth Z value or colour (yellow for peaks) to make a ridge or a peak pop just a bit more?

No, but can I use that please!? I can see how I would use masks (mapped 0 to 1) to control the effects.

As another aside, have you ever tried using gamma to emphasise perception of local features? I am mainly thinking of the perception of relief near sea level (but it could be relative to any local level ground). I adjust gamma of elevation to make low elevations seem more distinct than higher--since I think that is how we think.

Tim

Tim,

As another aside, have you ever tried using gamma to emphasise perception of local features?

Not in the past, but having just read your suggestion, I will be very shortly!

Thanks, that's very helpful.

The idea of letting each layer has its blending options is familiar, indeed. We could do it, yes. There is a drawback to it in that blending isn't terribly friendly to parallel rendering and various rendering optimizations (eg, it increases the proportion of work that has to be done without parallelism or with reduced parallelism), but so what if that's the right paradigm here.

We'll consider it.

Thanks!

Would some of the reduced parallelism result, in any case, from allowing more flexible Z-ordering, in particular defined ordering between objects within a layer (which I think is planned)?

Could users cope if rendering were partly speculative, and could therefore change on the fly? Or does each rendered view need to be right first time (even if that is slower)? I'm not sure.

[Added.] My hunch is that most users would find speculative rendering spooky or "flaky", if they could see it. So that might mean an intermediate buffer, followed by synchronous copy. Maybe this is already built in to OpenGL or DirectX, I am definitely out of my depth (partly occluded).

Flexible Z-ordering within a layer (which we want to allow, yes) will affect rendering within that layer only. What blending modes do is increase the amount of time the system has to spend merging the results of otherwise parallel rendering of multiple layers. With complex blending tunable on a per-layer basis, if we render layers L1, L2, L3, L4, L5 and L5 is the slowest layer rendering, we have to keep merging all of L1, L2, L3, L4, L5 every time we update the screen until L5 is finally rendered in full. When blending is common for all layers and is set to something simple like existing pixel / missing pixel or at most an opacity, L1, L2, L3 and L4 can be merged together even though L5 is still rendering, this saves both time and space. Now, this omits a lot of details both ways in that some optimizations are still possible even with complex blending and on the other hand some are still bad even without complex blending, but perhaps this illustrates the point well enough.

A question: when you have L1, L2, L3 with L1 being the top, L1 has a blending mode that combines with L2, and you turn L2 off, you'd want L1 to now combine with L3, correct? Because sometimes people want to keep L1 combining with L2 even though L2 itself is turned off, because combining with L3 makes little sense due to the nature of the combining.

You are a generous man. Thank you for trying to explain that. I think I follow (or will soon).

As to your question, I had never thought about this before, but I think the answer should be (not must be) the second, that L1 should keep combining with L2, even though L2 is off. Don't know, need to think harder.

The effect of turning off L2 should not feel like making a change of order. To do that, we should, exactly, make a change of order.

I think that's right. I will make some checks tomorrow to see what Photoshop does.

In a way it solves this issue, or further complicates it, by introducing layer groups (with optional knockout, shallow and deep... heavens to Betsy).

OK I would agree with your first thought, L1 blends with whatever is the result below it. If L2 is on, then the result of L2 (which may include content/effects from L3 and below). If L2 is off, then the result of L3 (and below if applicable).

I convinced myself like this: (1) turn on L1, L2, L3. (2) Now turn off L2. (3) Now drag L2 below L3. Step 2 should (may) make a visible change. Step 3 should not.

Cool, textures like this would be wild

https://github.com/mdsumner