Subscribe to this thread
Home - General / All posts - selecting multiple tags and export them
eldoland9 post(s)
#09-Jun-21 18:25

Hi,

im trying to select multiple tags (restaurants, bakeries, convenience stores, groceries, etc) in a specific city.

I followed this tutorial

https://manifold.net/doc/mfd9/index.htm#example__connect_to_an_osm_vector_server.htm

and this video

https://www.youtube.com/watch?v=XtEHL0tgXTk

how can i select mutliple tags?

and is there a way to export the results i get?

thanks!

Sloots

678 post(s)
online
#10-Jun-21 09:06

Try the following in the Select Pane => Expression:

StringJsonValue([Tags], 'amenity', true) IN ('restaurant', 'bar')

OR

 StringJsonValue([Tags], 'shop', true) IN ('supermarket')

Alternatively write a query with the same expression in the WHERE clause

SELECT

 *

INTO [subset of osm]

FROM

 [OSM]::[Osm Drawing]

WHERE

 StringJsonValue([Tags]'amenity'trueIN ('restaurant''bar')

OR

 StringJsonValue([Tags]'shop'trueIN ('supermarket');

ALTER TABLE [subset of osm] (

  ADD PROPERTY 'FieldCoordSystem.geometry' '{ "Axes": "XYH", "Base": "World Geodetic 1984 (WGS84)", "Eccentricity": 0.08181919084262149, "MajorAxis": 6378137, "Name": "Latitude \\/ Longitude", "System": "Latitude \\/ Longitude", "Unit": "Degree", "UnitLatLon": true, "UnitScale": 1, "UnitShort": "deg" }'

);

CREATE DRAWING [subset of osm Drawing] (

  PROPERTY 'FieldGeom' 'geometry',

  PROPERTY 'Table' '[subset of osm]'

);


http://www.mppng.nl/manifold/pointlabeler

Dimitri


7,413 post(s)
#11-Jun-21 09:30

In addition to what Sloots recommends, you can do this in point and click style with no SQL by repeatedly using the Select pane.

Select for one tag, like restaurant, using the Search template with the Action set to "replace selection", and then do it again for a different tag, like supermarket, changing the Action to "add to selection." That's the equivalent of doing a WHERE clause using OR, like WHERE [tag]='restaurant' OR [tag]='supermarket'.

You can then keep doing selections in the table to refine what you've selected by using other searches on whatever field you want, but changing the Action as desired to 'subtract from selection,' 'intersect with selection', or 'invert with selection."

You can then export just the selected records in the table with Edit - Export Selected.

eldoland9 post(s)
#12-Jun-21 12:47

Hi guys,

thanks! it is working!

However, i can't see the Edit - Export Selected

https://prnt.sc/155bco9

where is it?

Also, when he drags and drop the osm drawing in the video at 1,32 here https://www.youtube.com/watch?v=XtEHL0tgXTk

he gets everything in that view.

Can i get everything inside a boundary instead?

I mean, can i drag and drop the osm drawing inside the London boundaries for example?

thanks!

Dimitri


7,413 post(s)
#12-Jun-21 13:42

You're using Manifold Viewer, which is read-only. To export, you must use Release 9.

eldoland9 post(s)
#12-Jun-21 14:11

oh ok! makes sense!

is there a way in the viewer to only "count" the number of the locations i selected with the query?

thanks!

eldoland9 post(s)
#12-Jun-21 14:12

ok i think they are the "records" in the info panel

eldoland9 post(s)
#12-Jun-21 16:47

what im looking for is to estimate the number of restaurants, groceries, bakeries, pastry shops, pubs, (basically all food related businesses) in a specific city

i tried with Austin, but the results are really off (i think because when i drag and drop the osm drawing object it selects also the areas that are not really Austin)

is there a way to select only a city boundary and check the point of interests within that boundary?

thanks!

Dimitri


7,413 post(s)
#12-Jun-21 19:43

Seems like there are two things going on that that are a skill set challenge.

1. Working with OSM data. OSM is difficult to work with even for experienced GIS people because it is organized in a very complex way. So you have to invest in understanding how it is organized so you can get the info you want. Manifold has all the right tools to deal with all that, but you have to know what OSM is to get what you want out of it.

2. GIS skills in general and knowing how to do what you want in Manifold. If the data you were working with was more simply organized, you'd have one layer that was points of interest, with a field identifying what each point is (restaurant, grocery, etc.), and you'd have one or more other layers that had areas for the different jurisdictional areas, like, an area for the city of Austin, other areas for unincorporated county land, other areas for other cities/towns and so on.

That makes your task...

to estimate the number of restaurants, groceries, bakeries, pastry shops, pubs, (basically all food related businesses) in a specific city

... easy to accomplish. For example, you could select all the points of interest you wanted, copy them and make a new layer, and then do a spatial join using the Join dialog to add to each jurisdictional field the "count" of points of interest that fall within that jurisdiction. That would do the count operation for all the jurisdictions at once. Could do the entire US that way with one use of Join dialog.

Dimitri


7,413 post(s)
#12-Jun-21 17:32

is there a way in the viewer to only "count" the number of the locations i selected with the query?

and

ok i think they are the "records" in the info panel

The number of records is the total number of rows in that table. If you select, say, a dozen of those, that won't change the number of records in the info pane.

What you can do is select whatever it is you want to select, choose Edit - Copy (shortcut: Ctrl-C), and then in the Project pane do an Edit - Paste (Shortcut: Ctrl-V), to paste the selected records as a new table. Open the table and the info pane will tell you the number of records in that table. Those, of course, are the number of records that you originally selected.

eldoland9 post(s)
#12-Jun-21 18:02

ok yes i got it, thanks

however, it seems the number of locations is different based on what area is selected

for example this time i got this for Austin https://prnt.sc/1565q8u

but earlier it was much larger (and a perfect rectangle), so i really don't know how to drag and drop the osm drawing object into the map to get the osm data for a specific city boundaries

do you have any idea?

thanks!

eldoland9 post(s)
#12-Jun-21 18:27

ok, i think i got something

i got the locations for Austin and Portland on manifold based on osm and here are the results

Austin: 1172

Portland: 2212

this is way off because when scraping yelp i got

Austin: 6368

Portland: 7069

at this point i guess that osm is missing many many locations

is there a way to connect manifold to yelp or tripadvisor or gmaps or zomato or any other more reliable resource?

thanks again!

Dimitri


7,413 post(s)
#12-Jun-21 19:53

i got the locations for Austin and Portland on manifold based on osm

OK... how did you do that?

at this point i guess that osm is missing many many locations

In my experience, OSM has pretty good info for the US. More likely the difference between the OSM numbers you got for Austin/Portland and those you got from scraping yelp are caused by different selection methodology.

You can't drill into that without knowing details on exactly how you're selecting items of interest from OSM, and the details of how you're scraping data from yelp.

For example, if you want to count the number of restaurants OSM has within Austin, you're probably going to count within whatever are the city limits of Austin, that is, the actual jurisdictional boundaries. If you get a shapefile showing the city of Austin as an area, so you can do spatial joins, the area shown for Austin will be just for the actual city itself, not counting all the near-in areas that locals think of as "Austin."

If you ask Yelp for all the restaurants in Austin, I'd bet Yelp isn't going to worry about the actual jurisdictional limits but is going to fetch all the restaurants in unincorporated county land that abuts the legal boundaries of Austin, other towns that abut Austin, and all those other places that people think of as "Austin" that might technically not be within the city boundaries.

You get that effect all the time with urban areas. For example, when people want to make comparisons to "Boston" they're often surprised that Boston itself is a very small geographic region with much of what people think of as the city of Boston actually being different towns and cities that are part of the Boston urban area, like Cambridge, Allston, Brookline and similar places.

eldoland9 post(s)
#13-Jun-21 08:03

Hi Dimitri,

thanks for the explanation! yes this is why i was asking if i can drag and drop the osm drawing into a specific area (=the city boundaries).

So far i have used the "red thin line" gmaps shows when you look for Boston for example, but i have no idea how to tell manifold to drag and drop the osm drawing into that line

I got the results for Ausin and Portland following this tutorial https://www.youtube.com/watch?v=XtEHL0tgXTk

when dragging and dropping the osm drawing into the map i got this https://prnt.sc/158g7rw

then i added this code in the expression field in manifold

StringJsonValue([Tags], 'amenity', true) IN ('marketplace', 'bar', 'biergarten', 'cafe', 'fast_food', 'food_court', 'ice_cream', 'pub', 'restaurant')

OR

StringJsonValue([Tags], 'shop', true) IN ('alcohol', 'bakery', 'beverages', 'butcher', 'cheese', 'chocolate', 'coffee', 'confectionery', 'convenience', 'deli', 'dairy', 'farm', 'frozen_food', 'greengrocer', 'health_food', 'ice_cream', 'organic', 'pasta', 'pastry', 'seafood', 'spices', 'tea', 'department_store', 'kiosk', 'supermarket', 'wholesale')

OR

StringJsonValue([Tags], 'craft', true) IN ('brewery', 'caterer', 'confectionery', 'winery', 'distillery')

i got this for Austin https://prnt.sc/158fn9y

that looks like it is grabbing locations within the red line as showed in google maps https://prnt.sc/158fxa7

Is there a way i can tell manifold to map a larger area or maybe, is there a way to draw a polygon (even by hand) and tell manifold that i want that area mapped?

thanks!

Dimitri


7,413 post(s)
#13-Jun-21 19:32

I got the results for Ausin and Portland following this tutorial

OK. That doesn't tell us exactly what you did. The video you mention shows how to drag and drop an OSM web server layer into a map, to zoom far into Chartres, France, and to draw a rectangular selection box to select everything OSM within the square selection box, which covered a few dozen blocks in the zoomed-in view.

Using that procedure you're not going to select everything precisely within the city boundaries of Austin or Portland, since neither Austin nor Portland are rectangular shaped areas.

I think that's what you did, since your illustration at https://prnt.sc/158g7rw shows what looks like the result of a bunch of rectangular selections you made by hand in the Portland area.

i got this for Austin https://prnt.sc/158fn9y

that looks like it is grabbing locations within the red line as showed in google maps https://prnt.sc/158fxa7

The first illustration shows Portland, but in any event all you seem to be doing is grabbing OSM objects from within the rectangular shaped area you drew freehand with a map. That has no relationship to the borders of the towns. It's just selecting what you told it to select by drawing a selection rectangle free hand with the mouse.

The use of expressions to select amenity and shop tags will just pick out the named tags. If that's a comprehensive list of what you're interested in, that's good. But if there are other tags, like "brewpub" or "brewery", it won't pick out those.

Shifting gears a bit, I don't think you're taking the right approach using an OSM web server instead of the actual OSM data.

It's better to work with the actual OSM data and not a web summary. OSM is a very complicated data set. The web viewer presentation is a simplification. If you really want to work with OSM data, you should download and work with OSM PBF files. See the Manifold user manual topics for those. Viewer is not the right tool for that because PBF files are big. When you go to the hassle of downloading them and importing them, you will want to be able to save the Manifold project (which opens instantly thereafter), so you can continue working without having to a) hope you can do everything in only one session without ever closing it, or b) import the PBF again every time you want to start work.

Sources for real OSM data:

http://download.geofabrik.de/north-america.html

Easier than working with PBF is to use somebody else's extractions:

Global OpenStreetMap data extracts | osmdata.xyz (post processed extractions... very convenient. Amenity, Leisure, shop, and tourism extractions sounds like they might interest you)

Second, you need to learn more about the tool by reading the basics in the user manual. For example:

Is there a way i can tell manifold to map a larger area or maybe, is there a way to draw a polygon (even by hand) and tell manifold that i want that area mapped?

Yes to both, and there are topics that cover all that. I get the feeling that a lack of experience in GIS in general and in Manifold specifically is holding you back. It's quicker to bite the bullet and spend a day in structured, focused, application of RTFM skills to get the basics covered than it is to go back and forth for days not quite grounded. Start at the beginning of the User Manual (Read Me First) and read up through the end of the Basics chapter. Mix in Introductory topics from the Examples chapter to break up that steady stream of reading. That sounds like a lot but it goes quickly if you stay focused and just do it.

You should also do what I mentioned in my previous post: get a data set that shows the cities of Austin and Portland as area objects. Don't waste time drawing those freehand.

Stick with it and you'll do OK. It's clear from the expression examples you included that you're thinking logically and have the smarts to do this sort of work.

Hope that helps!

eldoland9 post(s)
#13-Jun-21 19:45

Hi Dimitri,

totally agree on everything, you are super right!

im a bit in a hurry and i wanted a quick solution but without knowledge im just wasting time!

i will definitely read all the basics!

in the meanwhile i found out geofabrik and downloaded some files to test them.

It goes without saying that ofc im struggling to find the "tags" there (for restaurant, groceries, bakeries, etc) and im stuck again

I will check the osmdata link you posted

thanks again for your patience!

Dimitri


7,413 post(s)
#14-Jun-21 06:58

OSM data in original form (as interchanged via PBF) is complex, for sure. Many people prefer to work with the post-processed extractions from osmdata.xyz (or similar sites).

The SQL Example: Extract Airport Runways from an OpenStreetMap PBF example topic might help.

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.