Subscribe to this thread
Home - General / All posts - Object transparency in M9
apo
168 post(s)
#12-Mar-22 18:05

The subject of transparency of objects in M9 has surfaced several times in the forum. The possibility of assigning a transparency not at the level of the layer, taken as a whole, but per object allows to value this transparency for objects which would be superimposed in the map. This functionality is vital for us, especially in thematic mapping with proportional symbols.

As the M9 renderer is capable of integrating transparency on objects but the colour setting interface does not really allow it (yes you can interpolate between transparent black and another colour but that's not what I call really setting an alpha value), here is a code that allows you to go beyond this limit. Thanks to SGA and its very interesting and quick coding, based on our discussions.

Here is an example of the results obtained with the code in M9 with several different cases of transparency application (view attached image and reads from the bottom line up)

  • the first line corresponds to an object layer without transparency
  • the second line corresponds to a layer with a general opacity of 80%, which illustrates the limit of transparency per layer, with objects masking others
  • the third line corresponds to a layer with transparency per object (80% opacity), which achieves the desired goal
  • the fourth line highlights the fact that transparency can vary (linearly) between colour classes
  • the fifth line shows that it can also be applied to a colour ramp.

at this point the code requires the drawing to have a color ramp already set, at it only manage to add the alpha values

VALUE @firstAlpha INT32 = 30;

VALUE @lastAlpha INT32 = 80;

VALUE @drawingName NVARCHAR = 'YourDrawingName';

-- Function to add alpha to any Manifold encoded color

FUNCTION ColorValueM9A(@m9 INT32, @a INT32) INT32 AS

  (CASE WHEN @a>=50 THEN Cast(128-(@a-50)*128/50 AS int32) WHEN @a<50 THEN Cast(-1*@a*128/50 AS int32) END)*256^3  + CASE WHEN Abs(@m9)>256^3 THEN (Abs(@m9/(256^3))-FLOOR(Abs(@m9/(256^3))))*256^3 ELSE @m9 END

END;

DROP TABLE alpha_test1_bnj4jk;

-- retrieve color scheme from mfd_meta and split colors classes

SELECT StringSubstringLen(Value,0,StringFind(Value,':')) as entityname,

Cast(StringSubstring(Value,StringFind(Value,':')+1) as int32) as entityval,

ColorValueM9A(Cast(StringSubstring(Value,StringFind(Value,':')+1) as int32),80) AS entitynewval,

 Value

INTO alpha_test1_bnj4jk

FROM

(

SELECT

SPLIT CALL StringToTokens(

    StringSubstring(Value ,

        StringFind(Value,'"Values": {' )

    +11)

',')

FROM (select * from mfd_meta WHERE Name like @drawingName AND Property LIKE 'StyleAreaColorBack')

) as D;

ALTER TABLE alpha_test1_bnj4jk (add mfd_id int32);

-- interpolate alpha value according to the number of classes of the color scheme

DROP TABLE alpha_test2_bnj4jk;

SELECT mfd_id, entityname, entityval, (@lastAlpha-@firstAlpha)*(mfd_id-1)/(max_mf-1)+@firstAlpha AS alpha

INTO alpha_test2_bnj4jk

FROM alpha_test1_bnj4jk

INNER JOIN (SELECT Max(mfd_id) as max_mf FROM alpha_test1_bnj4jk)

ON 1=1;

-- assign the new color including the alpha value

DROP TABLE alpha_test3_bnj4jk;

SELECT mfd_id, entityname, entityval, ColorValueM9A(entityval, alpha) AS newentityval

INTO alpha_test3_bnj4jk

FROM alpha_test2_bnj4jk;

-- generate the json code for StyleAreaColorBack

DROP TABLE alpha_test4_bnj4jk;

SELECT StringSubstringLen(Value, 0, StringFind(Value,'"Values": {')+11) & StringJoinTokens(StringConcat(entityname, ' : ', CAST(newentityval AS NVARCHAR)),',') & '}}' as newjson

INTO alpha_test4_bnj4jk

FROM alpha_test3_bnj4jk

INNER JOIN (select * from mfd_meta WHERE Name like @drawingName AND Property LIKE 'StyleAreaColorBack') as mm

ON 1=1

GROUP BY Value;

-- update the value in mfd_meta

UPDATE (SELECT m.mfd_id, Name, Property, m.Value, a4.newjson

FROM mfd_meta AS m

INNER JOIN (SELECT dname, prop, FirstNonNull(newjson) AS newjson FROM (SELECT @drawingName AS dname, 'StyleAreaColorBack' AS prop, newjson FROM alpha_test4_bnj4jk) AS sub GROUP BY dname, prop) AS a4

ON Name=dname AND Property=prop)

SET Value=newjson;

DROP TABLE alpha_test1_bnj4jk;

DROP TABLE alpha_test2_bnj4jk;

DROP TABLE alpha_test3_bnj4jk;

DROP TABLE alpha_test4_bnj4jk;

Attachments:
colour_alpha_tests.jpg

apo
168 post(s)
#14-Mar-22 13:43

Few more information to better describe the use of this query.

Only 3 parameters to be set being

drawingName : the name of the drawing with the colour ramp to be adjusted

firstalpha : the opacity percentage to be applied to the first colour item in the ramp

lastalpha : the opacity percentage to be applied to the last colour item in the ramp

remarks :

  • The alpha values for the in between colour items will be linearly interpolated.
  • For a constant alpha value just set the first and last using the same value

Finally to illustrate the use we are making of this code, just a use case map depicting bivariate symbols having the size of the circles proportional to the population and the colour ramp linked to the purchase power level of the corresponding population. Three cases of transparency are presented in parallel :

  • no transparency (left) which hides underlying layers and background as part of the density in some areas

  • layer transparency (center) which hides part of the density

  • object transparency (right) which both allow to see background and overlapping objects giving a sense of density.

    In a pure respect of the bivariate map semiology the Z order of the circles should observe a rank with the bigger to be behind the smaller ones, which is not yet handled by M9

    Attachments:
    colour_alpha_map.jpg

  • Sloots

    656 post(s)
    #14-Mar-22 15:16

    Thanks for this apo! Especially the conversion of M9 color code to M9 color with alpha. I noticed however there is a flaw when the alpha is set to 50. 49 percent and 51 percent are working fine. I noticed that 50 plays a key role in your function, but somehow things go wrong there.

    Just out of curiosity, where did you find info about this alpha color coding?

    Chris


    http://www.mppng.nl/manifold/pointlabeler

    apo
    168 post(s)
    #14-Mar-22 15:32

    Thanks to point this 50% problem, we will investigate further this problem which is linked to the encoding of 50% in 128. The encoding of the values of alpha is a bit strange. We back-engineered it applying an interpolation between transparent color to black and analyzing the color code which once done contains the alpha values. This showed us that the 256 alpha values are using something similar to

    0 up to 50% runs from 0 to 128

    51% to 100% runs from -1 to -128

    then for sure the 50% is somewhere lost in the mist of our explorations

    But we will for sure point it and correct the code.

    Good point Chris

    oeaulong

    503 post(s)
    #14-Mar-22 15:51

    clever approach. Thanks for this example.

    Sloots

    656 post(s)
    #14-Mar-22 19:29

    Here my little piece of script/sql to convert argb to the number that M9 requires. The C# part illustrates best what is going on behind the scenes, the SQL is faster (and easier to use in my opinion).

    SCRIPT [cs-alpha] ENGINE 'C#' [[

     

    class Script

    {

      static public int ARGB2INT32(int a, int r, int g, int b)

      {

        return a<<24 | r<<16 | g<<8 | b; 

      }

    }

     

    ]];

    VALUE @r INT32 = 128;

    VALUE @g INT32 =   0;

    VALUE @b INT32 = 255;

    FUNCTION ARGB2INT32(@a INT32, @r INT32, @g INT32, @b INT32) INT32 AS

      SCRIPT INLINE [cs-alpha]

      ENTRY 'Script.ARGB2INT32';

    SELECT

     value as alpha

     , @r as red

     , @g as green

     , @b as blue

     , ARGB2INT32(value, @r, @g, @b) AS via_cs

     CASE 

     WHEN value < 128 THEN (value * 256^3 + @r * 256^2 + @g * 256 + @b)

     ELSE (value * 256^3 + @r * 256^2 + @g * 256 + @b) - 2^32

       END AS via_sql

    FROM

     CALL ValueSequence(0, 255, 1)


    http://www.mppng.nl/manifold/pointlabeler

    apo
    168 post(s)
    #15-Mar-22 13:49

    Here is the second version with the following improvements :

    • manage single color style and not only ramps
    • manage correctly the 50%
    • correct a bug in handling 1 to 9% values because of the string sorting of values in the json
    • correct a bug which was corrupting colours under 50%

    -- $manifold$

    VALUE @firstAlpha INT32 = 60;

    VALUE @lastAlpha INT32 = 80;

    VALUE @drawingName NVARCHAR = 'Drawing 1';

    -- Function to add alpha to any Manifold encoded color

    FUNCTION ColorValueM9A(@m9 INT32, @a INT32) INT32 AS

    (CASE WHEN @a>50 THEN Cast(128-(@a-50)*128/50 AS int32) WHEN @a<=50 THEN Cast(-1*@a*128/50 AS int32) END)*256^3 + CASE WHEN Abs(@m9)>256^3 THEN Abs((@m9/(256^3)-FLOOR(@m9/(256^3))))*256^3 ELSE @m9 END

    END;

    DROP TABLE alpha_test1_bnj4jk;

    -- retrieve color scheme from mfd_meta and split colors classes

    SELECT *

    INTO alpha_test1_bnj4jk

    FROM

    (

    SELECT type, valuevalue,StringSubstringLen(Value,0,StringFind(Value,':')) as entityname,

    Cast(StringReplace(StringSubstringLen(Value,0,StringFind(Value,':')),'\"',''AS int32) as entityname_int,

    Cast(StringSubstring(Value,StringFind(Value,':')+1) as int32) as entityval,

    ColorValueM9A(Cast(StringSubstring(Value,StringFind(Value,':')+1) as int32),80) AS entitynewval,

    Value

    FROM

    (

    SELECT

    CASE WHEN StringFind(Value,'"Values": {' )>0 THEN 'class' ELSE 'unique' END AS type,

    Cast(StringSubstringLen(Value,

    StringFind(Value,'"Value": ' )+9,

    StringFind(StringSubString(Value,StringFind(Value,'"Value": ' )+9),',')

    AS int32) as valuevalue,

    SPLIT CALL StringToTokens(

    StringSubstring(Value ,

    StringFind(Value,'"Values": {' )

    +11)

    ',')

    FROM (select * from mfd_meta WHERE Name like @drawingName AND Property LIKE 'StyleAreaColorBack')

    as D

    ORDER BY entityname_int

    AS E;

    ALTER TABLE alpha_test1_bnj4jk (add mfd_id int32);

    -- interpolate alpha value according to the number of classes of the color scheme

    DROP TABLE alpha_test2_bnj4jk;

    SELECT mfd_id, type, valuevalue,entityname, entityval, (@lastAlpha-@firstAlpha)*(mfd_id-1)/(max_mf-1)+@firstAlpha AS alpha

    INTO alpha_test2_bnj4jk

    FROM alpha_test1_bnj4jk

    INNER JOIN (SELECT Max(mfd_id) as max_mf FROM alpha_test1_bnj4jk)

    ON 1=1;

    -- assign the new color including the alpha value

    DROP TABLE alpha_test3_bnj4jk;

    SELECT mfd_id, type, ColorValueM9A(valuevalue, @firstAlpha) AS newvaluevalue, entityname, entityval, ColorValueM9A(entityval, alpha) AS newentityval

    INTO alpha_test3_bnj4jk

    FROM alpha_test2_bnj4jk;

    -- generate the json code for StyleAreaColorBack

    DROP TABLE alpha_test4_bnj4jk;

    SELECT First(type) AS type,

    CASE

    WHEN First(type) LIKE 'class' THEN

    StringSubstringLen(Value, 0, StringFind(Value,'"Value": ')+9) & Cast(First(newvaluevalue) as NVARCHAR) & ', "Values": {' & StringJoinTokens(StringConcat(entityname, ' : 'CAST(newentityval AS NVARCHAR)),',') & '}}'

    WHEN First(type) LIKE 'unique' THEN

    '{ "Value": ' & Cast(First(newvaluevalue) AS NVARCHAR) & ' }'

    END as newjson

    INTO alpha_test4_bnj4jk

    FROM alpha_test3_bnj4jk

    INNER JOIN (select * from mfd_meta WHERE Name like @drawingName AND Property LIKE 'StyleAreaColorBack'as mm

    ON 1=1

    GROUP BY Value;

    -- update the value in mfd_meta

    UPDATE (SELECT m.mfd_id, Name, Property, m.Value, a4.newjson

    FROM mfd_meta AS m

    INNER JOIN (SELECT dname, prop, FirstNonNull(newjson) AS newjson FROM (SELECT @drawingName AS dname, 'StyleAreaColorBack' AS prop, newjson FROM alpha_test4_bnj4jk) AS sub GROUP BY dname, prop) AS a4

    ON Name=dname AND Property=prop)

    SET Value=newjson;

    DROP TABLE alpha_test1_bnj4jk;

    DROP TABLE alpha_test2_bnj4jk;

    DROP TABLE alpha_test3_bnj4jk;

    DROP TABLE alpha_test4_bnj4jk;

    rk
    603 post(s)
    #01-Jul-22 13:48

    I discovered that this style effect works 'per object'. It may not be precise enough for thematic mapping but very easy to apply as a quick hack.

    ALTER DRAWING [Drawing] (

      ADD PROPERTY 'StyleArea' '{ "Value": { "Stroke": "0.5", "Style": "line" } }',

     ADD PROPERTY 'StyleAreaColorBack' '{ "Value": -16777216 }',

     ADD PROPERTY 'StyleAreaSize' '{ "Value": 1.25 }'

    );

    Attachments:
    object_transparency.png

    rk
    603 post(s)
    #01-Jul-22 13:52

    Can we define a default style? Favorite styles? I want this to be my default

    Dimitri


    7,316 post(s)
    #01-Jul-22 14:26

    Very cool!

    lionel

    992 post(s)
    #02-Jul-22 07:49

    there is not 1 mandatory field for style but 2 : the second field is the one that define space distance between internal motif lines. This value should be set to 1,25 or 1,5 to not see the motif apply.

    -lower value and the motif rendering is black instead grey field

    -higher value and the motif show line or grid instead grey field

    grey colors can be change to any others colors by change the color of the border line edge = internal line !!

    Attachments:
    Transparency_manual-script.png


    Book about Science , cosmological model , Interstellar travels

    Boyle surface fr ,en

    lionel

    992 post(s)
    #02-Jul-22 08:14

    each time fill shape color overlap , the color is more opaque ?

    add black mean shade but I don't know if it is the right word to use to describe add transparency layers of same color ?

    Attachments:
    SQL_color_Style.map
    tranparencyLayer.png


    Book about Science , cosmological model , Interstellar travels

    Boyle surface fr ,en

    Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.