Geocoding Data Sources

The Geocoding Tools package is an optional extension to Manifold System that provides street address geocoding capability as well as access to the Manifold Geocoding Database data provided on the Manifold downloads site for both US street address geocoding as well as the easy creation of drawings showing US streets. The Geocoding Tools package also requires access to one or more geocoding data sources that are appropriate for the geographic region of interest.

 

Very important: the Geocoding Tools package will not be operational until you install at least one geocoding data source. You must install at least one geocoding data source.

 

A geocoding data source is a data source that provides the internal Manifold geocoding engine information about the location of street addresses. The geocoding engine can consult the data source to get information that helps the geocoding engine estimate the location of a particular street address. Once enabled by the installation of the Geocoding Tools package, Manifold can utilize the following geocoding data sources:

 

·      The Manifold Geocoding Database provided for free download on the manifold.net site. It provides data for streets in the United States extracted from the US Bureau of the Census TIGER database. The Manifold Geocoding Database is fast, but is not as complete as the internal databases within Microsoft's MapPoint product. It is limited to the US only. The Manifold Geocoding Database is provided at no charge on the manifold.net site, so it is usually the first geocoding data source employed by new users in the US and more often than not the only geocoding data source users in the US ever need employ.

·      Microsoft's MapPoint product in North American or European editions, which provide data either for North America or for selected countries in Europe. When MapPoint is installed on the same computer as Manifold System, the Manifold geocoder can reach into MapPoint's internal database to use it as a geocoding data source. MapPoint has one of the very best geocoding databases in the GIS industry, but it is significantly slower than using the Manifold Geocoding Database. Although MapPoint must be purchased, the combination of Manifold together with MapPoint is by far the least expensive way of doing street address geocoding for Canada and Mexico or for the countries supported by the MapPoint European edition.

·      User-provided geocoding data extensions in either address range or points of interest formats. User-provided data sources must be used outside of the US or of the coverage provided by MapPoint. Even within the US or Europe, user-provided data sources are often used when exact precision is required since only "points of interest" formats can provide exact address precision.

·      Web-based Manifold geocoding servers that provide geocoding services. Any such data source must fulfill the requirements of the Manifold Geocoding Server interface, but web services vary widely in their speed, accuracy and cost. Web services do have the potential advantage of greater timeliness, as it is possible to update a centralized data source more rapidly than, say, MapPoint editions may be updated. Visit the manifold.net web site for information on free geocoding server modules that may be downloaded.

 

Any combination of the above geocoding data sources may be used. This topic briefly surveys the above geocoding data sources and discusses the Manifold Geocoding Database in greater detail. See the specific Help topics for each of the other geocoding data sources for greater details.

 

A Geocoding Data Source Strategy

 

Users operating in the United States will normally use the Manifold geocoding database by default, since it is free. Users desiring slightly greater "hit" rates will also install MapPoint so MapPoint can be consulted if an address cannot be found in the Manifold geocoding database (the default setting).

 

Users in Canada or in the countries supported by MapPoint European edition don't really have much choice but to use MapPoint as a geocoding data source as this is generally far less expensive and easier to do than to try to acquire geocoding databases from other vendors and to convert such databases into the format required for user-provided geocoding data extensions.

 

Users outside the above areas cannot engage in street address geocoding unless they provide their own geocoding data extensions. That can be a difficult and expensive proposition. However, using Manifold on a notebook computer together with a portable GPS receiver, two employees in a vehicle (one to steer the vehicle and one to operate the notebook computer) can acquire addresses in points of interest format very rapidly. Many hundreds of streets can be done in a week and thousands of streets in well under a month by two people in a single vehicle. Once acquired, such data can be very valuable, the value of the data far exceeding the cost of acquiring it.

 

Web-based geocoding servers are so varied as to defy characterization. They range from very slow to very fast, from free to very costly, from very timely to remarkably obsolete. The Manifold Geocoding Server interface allows users to write modules that enable use of a very wide range of web-based geocoding servers.

 

Precision

 

Most users in the US doing street address geocoding with Manifold will use the Manifold US streets geocoding database provided as a free download on the manifold.net site. This database is based on the US government's official address database published in the TIGER/Line data set and allows the Manifold geocoder to find estimated positions for street addresses in the United States. It works best in urban and suburban areas where street addresses follow reasonably regular patterns.

 

Very Important: The Manifold Geocoding Database does not capture all possible street addresses nor can it guarantee the exact location of any address. Therefore, it must not be used for applications, such as 911 or other emergency response applications, which require every address to be exactly located. Note that the Manifold System End User License Agreement (EULA) specifically excludes uses, such as emergency response, that require fail-safe performance.

 

Nonetheless, the Manifold geocoder working together with the Manifold Geocoding Database is a good choice for many GIS tasks, such as demographic or marketing studies, that require geocoding of a reasonably high percentage of addresses. For such uses the Manifold Geocoding Database is highly effective and a spectacularly good value.

 

If Microsoft MapPoint is installed, the MapPoint database provides slightly greater accuracy in the United States, finding a few percentage points more addresses in large geocoding projects than is possible using the Manifold Geocoding Database. The MapPoint geocoder uses a variety of data sources in addition to government data to provide slightly better address recognition than is possible with government data only.

 

Some users prefer to use a Geocoding Data Server like the open source geocoding server module for Microsoft's Virtual Earth geocoding server. That has the advantage of frequently updated data in Virtual Earth, but it is slower since it has to go through Internet to geocode each address.

 

User-provided geocoding data extensions are normally used to provide custom information on street addresses not found within the Manifold Geocoding Database or MapPoint. For example, users may have specific address information licensed from commercial sources that provide more recent data than the TIGER data used in Manifold's geocoding database product or user may have custom address information for countries not covered by either Manifold or MapPoint.

 

Installing the Manifold Geocoding Database

 

Street address geocoding functionality within Manifold requires installation of an appropriate data source. If US street address geocoding is intended, we can use the Manifold Geocoding Database provided as a free download on the manifold.net site as our data source. If the database is not installed on the computer system and no other geocoding data source is provided, Manifold's street address geocoding commands and functions will not be available. The full US streets Manifold geocoding database requires approximately 950 MB of free space on disk.

 

Installing the Manifold US geocoding database:

 

1.    Download the Manifold Geocoding Database zip file from the product downloads page in the manifold.net website. This is a very large file, over 800 MB in size. If your Internet connection is not totally reliable it might get damaged in transit. If it does not unzip correctly, download it again until it does. Unzip the zip file.

2.    In Windows Explorer, drill down into the unzipped files to find the GCDB.msi installation program and double click on the GCDB.msi installation program to launch it. Note: If Windows Explorer has been set to hide extensions for well-known file types, this file may be listed in Windows Explorer as GCDB without the .msi extension. Warning: The geocoding installation .msi file is a huge file. Microsoft Windows Installer wants to check each .msi file before installing it so there may be a very long period of apparent inactivity while the Installer checks the GCDB.msi file, perhaps even tens of minutes on a slow machine. This is normal: just let the installation process continue to run and all will be well.

3. The installer will offer to install the geocoding database into a default folder, C:\Program Files\Manifold System\GCDB, within the default installation folder for Manifold System. If you have installed Manifold System in a different location on your hard disk it is strongly recommended that you update the file locations for geocoding in any IMS config.txt file so that geocoding capabilities will be available within Manifold IMS.

4. Launch Manifold. In the Tools - Options - File Locations pane, specify the folder used to install the geocoding database for the Geocoding Database folder and press OK. This step may be skipped if the default location for installation is used.

 

The states.dat file must always be available on your hard disk. The geocoding database is organized by US states with a file for each state. Each state file ends in a .dat extension and is named using the state postal abbreviation. For example, the geocoding database files for California and New York are ca.dat and ny.dat respectively.

 

The GCDB.msi installer will install geocoding database files for all US states. However, only those state files for which addresses will be geocoded need be located on the hard disk. Files for states that will not be used may be removed to free up disk space. For example, if we will be geocoding street addresses only in the state of California and no other state, we may delete all files from the GCDB folder on hard disk except the states.dat file and the ca.dat file.

 

Tech Tip: Don't delete any of the files. In an era of hard disks of virtually limitless size, there's little point in trying to save a few megabytes by deleting state files and at the same time much risk of inconvenience should those state files be needed at some future date.

 

To uninstall the US geocoding database and entirely remove all files, we may use the Windows Control Panel Add / Remove Programs applet. Removing the geocoding database will render all geocoding functions inoperative.

 

Geocoding Data Extensions

 

User extensions to street address geocoding data may be provided in the form of a Manifold .map project file that contain tables with columns using predefined names and types. The project file must be placed in the folder cited in the Tools - Options - File Locations pane for Geocoding Extensions and the Use geocoding extensions checkbox must be checked in the Tools - Options - Geocoding pane.

 

User geocoding extensions are always considered first when processing data. Next, either the Manifold Geocoding Database or MapPoint geocoding data are consider depending on whether MapPoint is set to be used before or after the Manifold geocoding database as specified in the Tools - Options - Geocoding dialog.

 

See the Geocoding Data Extensions topic for details on user-supplied extensions.

 

See Also

 

About Geocoding

Geocoding Tools

Street Address Geocoding

Geocoding with MapPoint

Geocoding Data Extensions

Manifold Geocoding Servers