Hi All,

I'd like to use Manifold 9 to run a query to process (create vegetation canopy out of partially classified .las files) roughly 2 TB of lidar data. The data is in las files on an external drive. Rather than merging the lidar and running the query (too much data), I'd like to process each las file and send the result to a table in Manifold.

I've attached a sample project file with the basic structure and some ultra simple data. I was thinking of creating a table that lists all the .las file names. Then the query would run, looping through each file name. Not sure if this is the best approach and my scripting skills are very lame. I couldn't find any scripts to copy to try a piece together a solution. Appreciate any help.

Attachments:
test 2.map

Hi Mike,

I had the same questions last week and made some tests and codes that are published in this post

I used a loop query (calling a script) to import and export data after process.

I published once the code for exporting a component.

In order to import a component it is a bit more complex because the import port sends the data in a temp database and then you have to report the table to your project. I just used this last week to import images

a.

Thanks for the tips!

Mike I use queries such as the example I posted below to achieve automation through SQL. I have adapted the steps to complete many different applications where you are repeating task steps on different components within the project or within a folder. It is very powerful to have access to mfd_meta!

https://georeference.org/forum/t163176.4#163179

Landsystems Ltd ... Know your land | www.landsystems.co.nz

Interesting query Dan. Never would have thought to try and use mfd_meta that way. Thanks!

here is the import approach I used

first a SQL call of two functions, the first one to prepare the target data (names etc.) and the second to import and inject the data

DROP TABLE [ROCK_RAW];

DROP IMAGE [ROCK_RAW IMG];

FUNCTION prepareimportcomponent(@imppath NVARCHAR, @zone NVARCHAR) NVARCHAR AS SCRIPT [S PREPARE IMPORT] ENTRY 'Script.PrepareImportComponent';

VALUE @ipt1 NVARCHAR = prepareimportcomponent(@p,CAST(@zone as NVARCHAR));

FUNCTION importcomponent(@imppath NVARCHAR) NVARCHAR AS SCRIPT [S IMPORT] ENTRY 'Script.ImportComponent';

VALUE @ipt2 NVARCHAR = importcomponent(@ipt1);

the code of the first function requires to parameters to be defined in the call @p the string contaning the fullpath to file to be imported and @zone the name of current zone defined by the loop

// C#

using System;

using System.IO;

class Script

{

public static string PrepareImportComponent(string imppath,string zone)

{

    Manifold.Application app = Manifold.Application;

        // copy proj file and rename it, this step is needed for me to set the proj syst based on the former file

        string impfilename = imppath;

        impfilename = impfilename.Replace("C:\\path\\","");

        impfilename = impfilename.Replace(".png","");

        app.Log(impfilename);

        string reffile = "C:\\path\\DEM_"+zone+".grd.mapmeta";

        string newfile = "C:\\path\\ROCK_"+zone+".png.mapmeta";

        if(File.Exists(newfile)==false){

            System.IO.File.Copy(reffile, newfile);

        }

        // rename file, this part is only because the external process generate its own filename and I want it back in my realm

        string rockreffile = imppath;

        app.Log("current name : "+rockreffile);

        string rocknewfile = "C:\\path\\ROCK_"+zone+".png";

        app.Log("new name : "+rocknewfile);

        System.IO.File.Copy(rockreffile, rocknewfile);

        return rocknewfile;

}

static Manifold.Context Manifold;

static void Main()

{

}

}

the second script is targeting the import using the fullpath to the targeted file to import

// C#

class Script

{

public static string ImportComponent(string impfile)

{

    string filename = impfile.Replace("C:\\path\\",""); 

    filename = filename.Replace(".png",""); 

    Manifold.Application app = Manifold.Application;

   // define target database

    Manifold.Database targetDb = app.GetDatabaseRoot();

   //import file in the sandbox database 

    Manifold.Database sourceDb = app.CreateDatabaseForFile(impfile, true);

   //retrieve imported component and its schema

    Manifold.Table source = sourceDb.Search(filename+" tiles");

    Manifold.Schema sourceSchema = source.GetSchema();

   //pipe the all imported content in my project

    string targetName = targetDb.Insert("ROCK_RAW", "table", sourceSchema);

    Manifold.Table target = targetDb.Search(targetName);

    Manifold.Schema.FieldSet fields = sourceSchema.Fields;

    Manifold.Sequence reclist = source.SearchAll(new string[] {"X","Y","Tile"});

    target.InsertBatch(reclist);

      app.Log("Done");

        return filename;

}

static Manifold.Context Manifold;

static void Main()

{

}

}

to close the import process, in the case of images, I need to set the projection correctly both in the table as in the drawing using SQL, and reset the tiles pyramid

ALTER TABLE [ROCK_RAW](

    ADD PROPERTY 'FieldCoordSystem.Tile' @adjustedprojsyst,

    ADD PROPERTY 'FieldTileSize.Tile' '[ 128, 128 ]',

    ADD PROPERTY 'FieldTileType.Tile' 'uint8',

      ADD INDEX [X_Y_Tile_x] RTREE ([X], [Y], [Tile] TILESIZE (128, 128) TILETYPE UINT8),

      ADD PROPERTY 'IndexCacheField.0.X_Y_Tile_x' 'X',

      ADD PROPERTY 'IndexCacheField.1.X_Y_Tile_x' 'Y',

      ADD PROPERTY 'IndexCacheField.2.X_Y_Tile_x' 'Tile',

      ADD PROPERTY 'IndexCacheFieldOptsValue.2.X_Y_Tile_x.TileSize' '[ 128, 128 ]',

      ADD PROPERTY 'IndexCacheFieldOptsValue.2.X_Y_Tile_x.TileType' 'uint8',

      ADD PROPERTY 'IndexCacheType.X_Y_Tile_x' 'rtree'

); 

CREATE IMAGE [ROCK_RAW IMG] (

  PROPERTY 'Table' '[ROCK_RAW]',

  PROPERTY 'FieldTile' 'Tile',

  PROPERTY 'FieldX' 'X',

  PROPERTY 'FieldY' 'Y',

  PROPERTY 'Rect' '[ 0, 0, '+ CAST((@resampx-2) * @dimx AS NVARCHAR) +', '+ CAST((@resampY-2) * @dimY AS NVARCHAR) +' ]'

);

TABLE CALL TileUpdatePyramids([ROCK_RAW IMG]);

Thanks for scripting examples. Much appreciated!