Subscribe to this thread
Home - General / All posts - Working with Individual Pixel Values In Image Components
joebocop
514 post(s)
#14-Sep-17 23:07

I would like to view the value of an individual pixel in an image when I alt-click on it, similar to how vector features' records are displayed in the Contents pane. Is this possible, in Manifold Future?

The image component is tiled (64x64, single channel uint8), so I am not able to get data directly from the individual pixel by those means.

  • Is the best approach here to re-tile the image component so that 1 tile = 1 pixel? Is there a better strategy?
A table component in the same project contains fields for the various pixel values and their "class" names.

  • Short of creating a new column in the image table and using a query to populate its values through a one-time join, is there a way to "relate" a table component to another table the way that was possible in Manifold 8?

Separately, when I zoom "way in" on my image components, the display seems to be applying a sort of interpolation, obscuring the individual pixels themselves and their boundaries.

  • Is there a way of disabling that display interpolation, or changing to an alternate rendering?

Thanks for the hand-holding.

Dimitri


7,413 post(s)
#15-Sep-17 05:24

similar to how vector features' records are displayed in the Contents pane. Is this possible, in Manifold Future?

No. See the discussion in the Contents - Record topic.

  • Is the best approach here to re-tile the image component so that 1 tile = 1 pixel? Is there a better strategy?

No, and Yes. No, don't make a tile of one pixel. Interrogate individual pixels using SQL functions like the various TileToValues functions.

is there a way to "relate" a table component to another table

The "relate" thing in 8 is just a different interface way of doing the Radian thing, but the Radian way using queries, joins, and computed fields is much better. It's true something that is less capable that you know how to use will be easier at first, but invest the time to learn the new way and you'll find it is much better.

Is there a way of disabling that display interpolation

People often want smoother displays and not literal representations of data. That's why you get things like anti-aliasing, interpolation between intermediate levels and further interpolation past the last level. It's also why there are "lossy" formats like MrSID and such in use, which are a form of interpolation using less data than the original.

Radian, and Future, go the "smoother display" route since on the balance that is the priority for people viewing rasters. As more and more capabilities are merged into Future and pixel level editing gets merged in, then, of course, you'll want to see individual pixels locked into the display, to grow big and blocky as you zoom in and not be smoothed, as Release 8 does, to enable clicking an individual pixel if that is desired.

Important: the reason there is a Manifold Future available to everyone and not just to a limited group of testers is to give people the opportunity to guide the product. The way to do that is well documented in the Suggestions page. The forum is for free-wheeling discussion. Suggestions guide the product.

joebocop
514 post(s)
#15-Sep-17 05:57

That's all good info, thank you.

Quickly, on the topic of suggestions, I submitted 3 last night and will continue to do so for sure. I hope my forum postings aren't misconstrued as complaints or gripes, or "suggestions" even; I'm legitimately enthused about this product and am trying to learn its features as quickly and thoroughly as possible with the expectation that it will be as revolutionary for my workflows as my discovery of release 8 was back in 2010. Part of creating "razor sharp" suggestions is making certain that, despite careful reading of manual topics, I haven't missed an expert technique or function that already exists in the product.

Again, thanks for the continued help.

Dimitri


7,413 post(s)
#15-Sep-17 06:21

I hope my forum postings aren't misconstrued

No, no misconstruing. Your posts and suggestions are great and very helpful to all. Keep them coming! :-)

When I write replies I have in mind that other people are reading the threads so I'll usually write comments with that in mind. Hope that helps others.

joebocop
514 post(s)
#15-Sep-17 22:27

Further on this, if I have a drawing component having points, and an image component whose individual pixel values I would like "joined" to that drawing's table based on their spatial coincidence ("touching", I suppose), is this doable via a query in Future?

The Release 8 equivalent that I can think of, though not via query component, is the "Transfer Heights" dialog, of course.

Still wrapping my head around the "tile" functions. Thanks.

tjhb
10,094 post(s)
#16-Sep-17 00:33

Yes it is possible now.

If you could supply some sample data I'd like to post an example (good practice).

joebocop
514 post(s)
#16-Sep-17 04:44

I've attached a Manifold Future map file with an image component and some points.

Thanks for the pointers.

Attachments:
pts_val-9.map

tjhb
10,094 post(s)
#16-Sep-17 05:14

That's great example data, thanks for posting it. Data always focusses things.

In this case we want a fast way to transfer pixel values to a comparatively small number of points, avoiding interpolation since the image data is in discrete classes.

joebocop
514 post(s)
#16-Sep-17 05:45

Yeah, exactly. I would like to be able to write a query that returns all the records in [p] with all their fields, along with one additional field containing the value of the pixel (class) from the image.

I just also realized that I don't know how to do this in release 8, besides using the Transfer Heights dialog.

Also, am I going to "link" a drawing to this query in order to get a Drawing representation of its results? That's my Release 8 brain working there; I'm thinking that the table needs to be "materialized" and then a Drawing created from it in Future, right? Like the query would be a SELECT INTO thing?

tjhb
10,094 post(s)
#16-Sep-17 06:04

It's very easy in Manifold 8 SQL (with a caveat). Have a look at Raster Extensions > Height(s, g) and Height(s, x, y). They are fast and good. The caveat is that I don't remember whether those functions use interpolation. I think that they don't, while near-equivalent Surface Transform functions do, but I could be remembering wrongly and would need to check.

The query in SQL9 will be a series of statements that does everything, including making a new table and a drawing attached to it. (It can also do formatting if that is required.) The query will probably use at least one script function, for efficiency, even though everything can be done in the new SQL.

I'll be trying to learn from Adam.

It's Saturday evening here, I'll get onto the code tomorrow morning.

tgazzard
146 post(s)
#16-Sep-17 05:56

Tim, I am looking forward to seeing how this can be done. I can see the advantage of tiles and of tables, but I haven't worked with that format before. Will be great to see a worked example

tjhb
10,094 post(s)
#18-Sep-17 09:43

(Still working on this, as simply as possible. Back tomorrow.)

tjhb
10,094 post(s)
#20-Sep-17 03:40

(Done now, not fully tuned. Back soon.)

tjhb
10,094 post(s)
#20-Sep-17 08:39

--SQL9

FUNCTION Converter(source TABLE, target TABLE)

    -- arguments must be display components (e.g. drawing, image)

    -- not source tables

    TABLE AS

    CALL CoordConverterMake(

        ComponentCoordSystem(target),

        ComponentCoordSystem(source)

        )

    END;

FUNCTION TileSizeXY(metadata TABLE, table_name NVARCHAR, field_name NVARCHAR)

    -- dimensions of tiles in named field and table

    INT32X2 AS

    (

    SELECT CAST([Value] AS INT32X2-- source string in form '[ x, y ]'

    FROM metadata

    WHERE StringToLowerCase([Name]) = StringToLowerCase(table_name)

    AND StringToLowerCase([Property]) = StringToLowerCase('FieldTileSize.' + field_name)

    )

    END;

FUNCTION PixelCoordXY

    -- coordinates of pixel in image space

    (tileX INT32, tileY INT32,

    pixelX INT32, pixelY INT32

    tilesizeXY INT32X2

    )

    INT32X2 AS

    VectorMakeX2(

        tileX * VectorValue(tileSizeXY, 0) + pixelX,

        tileY * VectorValue(tileSizeXY, 1) + pixelY

        )

    END;

FUNCTION VectorFloorX2(v FLOAT64X2)

    -- round vector values downwards

    INT64X2 AS

    VectorMakeX2(

        Floor(VectorValue(v, 0)),

        Floor(VectorValue(v, 1))

        )

    END;

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

-- temp database and reference to project

CREATE ROOT [Scratch];

USE ROOT [Scratch];

CREATE DATASOURCE [Project] AS ROOT;

--------------------------------------------------------------------------------

-- temp tables

CREATE TABLE [image points] (

    [point id] INT64

    [Geom] GEOM,

    [imageXY] FLOAT64X2,

    [pixelXY] INT32X2,

    INDEX [pixelXY_x] BTREEDUP ([pixelXY])

    );

CREATE TABLE [pixel values] (

    [pixelXY] INT32X2,

    [Value] INT32,

    INDEX [pixelXY_x] BTREE ([pixelXY])

    );

--------------------------------------------------------------------------------

-- output table and drawing

CREATE TABLE [Project]::[Points with image values Table] (

    [mfd_id] INT64,

    [source id] INT64,

    [Geom] GEOM,

    [Value] INT32,

    INDEX [mfd_id_x] BTREE ([mfd_id]),

    INDEX [Geom_x] RTREE ([Geom]),

    PROPERTY 'FieldCoordSystem.Geom' ComponentCoordSystem([Project]::[p])

    );

CREATE DRAWING [Project]::[Points with image values] (

    PROPERTY 'Table' '[Points with image values Table]',

    PROPERTY 'FieldGeom' 'Geom'

    );

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

-- populate temp tables

INSERT INTO [image points]

    ([point id][Geom][imageXY][pixelXY])

SELECT

    [mfd_id][Geom],

    [imageXY],

        -- exact location in image space

    CAST(VectorFloorX2([imageXY]AS INT32X2)

        AS [pixelXY]

        -- index of containing pixel

    -- reprojected point may be outside image bounds

    -- (with positive or negative coordinates)

FROM

    (

    SELECT

        [mfd_id][Geom],

        CoordConvertPoint(

            CALL Converter([Project]::[p][Project]::[t]), 

            GeomCoordXY([Geom], 0)

            ) AS [imageXY]

    FROM [Project]::[p]

    )

;

--------------------------------------------------------------------------------

INSERT INTO [pixel values]

    ([pixelXY][Value])

SELECT

    --[tileX], [tileY], [X], [Y], 

    --[tileXY],

    PixelCoordXY(

        [tileX][tileY],

        [X][Y]-- pixel in tile

        [tileXY]

        ) AS [pixelXY]-- pixel in image

    --[Value] -- FLOAT64 regardless of source type

    CAST([Value] AS INT32-- source type

FROM

    (

    SELECT

        [X] AS [tileX][Y] AS [tileY],

        TileSizeXY(

            [Project]::[mfd_meta],

            StringTrim(ComponentProperty([Project]::[t]'Table'), '[]'),

            ComponentProperty([Project]::[t]'FieldTile')

            ) AS [tileXY],

            -- calculate once before SPLIT

            -- (with copy after)

    SPLIT CALL TileToValues([Tile])

    FROM [Project]::[t]

    )

-- prefilter may be worthwhile for a very large image

--WHERE PixelCoordXY(

--    [tileX], [tileY],

--    [X], [Y], -- pixel in tile

--    [tileXY]

--    ) IN

--    (SELECT [pixelXY] FROM [image points])

;

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

-- write output table

INSERT INTO [Project]::[Points with image values Table]

    ([source id][Geom][Value])

SELECT

    t.[point id][t].[Geom],

    --t.[imageXY], 

    --t.[pixelXY],

    --u.[pixelXY],

    u.[Value]

FROM

    [image points] AS t

    LEFT JOIN

    [pixel values] AS u

    ON t.[pixelXY] = u.[pixelXY]

;

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

-- clean up

DROP TABLE [image points];

DROP TABLE [pixel values];

DROP ROOT [Scratch];

Attachments:
Pixel values to points b.txt
pts_val-9 e.map

joebocop
514 post(s)
#20-Sep-17 18:15

... and my education continues.

So, thank you.

I'll get this running in my env and report back.

tjhb
10,094 post(s)
#21-Sep-17 00:25

Some comments.

(1) I think the code above is too long, or anyway looks too long. It tries to be simple--part of that is breaking things up into manageable chunks--but the overall effect is verbose. Maybe a version more like what we would do in Manifold 8 (more nested queries, no temporary tables, no functions) would be just as good.

(2) Questions are welcome, from anyone who wants to wade through it. Even dumb ones, or hard questions I can't answer.

(3) I haven't done anything about multi-threading above. No THREADS directives at all. I need to experiment on a much larger image and/or more points.

(4) For the same reason, I don't know whether adding BTREEDUP and BTREE indexes on the [pixelXY] fields in the two temporary tables really makes the join faster. That's the only reason for adding them, but I don't know whether they help before testing on large data.

(5) The code above unpacks the whole image, to get the XY index of each pixel in image space (as opposed to tile space)--which is the basis for the join to reprojected points. That seems inefficient. The commented-out prefilter is a first guess at how to improve on that, especially for a large image. But there is probably a better way.

Instead of joining each point to the corresponding pixel in the space of the whole image (requring full unpacking), we could instead join each point to the containing tile (without unpacking), then extract the corresponding pixel for that point from the tile.

That seems more efficient if the number of points is relatively small, bearing in mind that fetched tiles, which might be shared by multiple points, will be cached.

I think this will become more important (a) with a large image, and especially (b) if we want interpolated values (therefore several pixels) rather than just the containing pixel.

adamw


10,447 post(s)
#21-Sep-17 13:44

Several comments:

The BTREEDUP indexes in [pixelXY] do make the LEFT JOIN faster. LEFT JOIN ends up using just one index (on the right side) but it is just simpler to be creating an index on each side because if you somehow end up being able to convert the task into an inner join, it will use both indexes to gain even more speed.

Taking each point, locating the tile for it, then getting the pixel is absolutely the way to go. The number of points is usually much less than the number of pixels and comparable with the number of tiles. If we compare the two techniques - (a) do a join on pixels vs coords, or (b) find a tile and then a pixel for each coord - with different numbers of points in an average tile, then if the average number of points per tile is low (like, 3) the join is going to lose big, then as the average number of points grows the join will be losing less and less and at some point it will eventually win, but I would expect the threshold to be pretty high. The second technique also gets easier benefits from multiple threads, because finding a tile and then a pixel does take some time.

Last, we will obviously add functions to take a geom, put it onto a raster and return some aggregate of the pixels (including a function to return all pixels as a table to process using a custom aggregate).

tjhb
10,094 post(s)
#22-Sep-17 05:16

Thanks so much for advice here Adam, it really helped.

Version 2 is just about done, but I need to test some index things before posting (since it’s still long, though happily, shorter than version 1).

tgazzard
146 post(s)
#10-May-18 03:01

Hi tjhb,

Is it possible to share version 2? Have some NetCDF files that require some extraction of values.

tjhb
10,094 post(s)
#10-May-18 07:34

I don’t know.

Please share data first, with a fully and exactly specified task.

It might be shareable, it might be billable. Let’s see.

tgazzard
146 post(s)
#10-May-18 09:45

Will see if I can post a file. Will have to check around privacy of the files.

I have another way to extract the values using python.

Was curious around speed + interested in learning how this would work in m9.

What you outlined above was useful and will work for the files in question.

Unfortunately no budget for this type of thing.

tjhb
10,094 post(s)
#21-Sep-17 01:10

One important thing I meant to add.

As always, the best way to read and digest the query above is by selecting various parts of it, then using Alt-Enter to run just that part. E.g. an individual statement, or a subquery. That is by the way the reason for putting subqueries between isolated parentheses like this

...

FROM

    (

    <subquery>

    )

...

So we can just select the <subquery> text and press Alt-Enter to run that and see the result. This is great for testing, also for understanding part by part.

(BTW Alt-Enter is also a great help for isolating ubiquitous 'Invalid object reference' errors, if code is written in executable chunks.)

As always, functions at the start of the query must be compiled before the main statements are run. Functions are always (re)compiled if the query is run as a whole, but if you are running individual statements (or parts of statements), you have to explicitly compile the functions once beforehand. To do that, the same thing: select all the FUNCTION text, then press Alt-Enter to "run" them (compile them).

If you edit a function, make sure to recompile it, by the same means. Compiled functions remain in state until recompilation (or until the query window is closed). (This is good: each succesive call to a function uses the cached version.)

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.