Visual Crossing Blog

A Primer on Shapefiles - part 2

In a tool such as OpenJump you can move geographic features and also edit attribute data.

In part one of this Shapefile series we discussed the basics of Shapefiles, their structure, and how to avoid corrupting them if you need to edit them. In this next part of the series we will discuss how to add shapefiles into the Visual Crossing environment.

Background

Since Shapefile is the inductor-standard GIS vector format, it is no surprise that Visual Crossing can read and manipulate Shapefiles. In fact, Shapefiles can be used in Visual Crossing as essentially a native file format. This means that Shapefiles can be copied directly into the Visual Crossing metadata without worrying about format conversions and data loss that is common when coercing one format for use on a different platform. In fact, when Visual Crossing does region aggregation, for example, it outputs the result as a Shapefile and stores it into the metadata directly. Thus if you need to use a 3rd-party tool to look at your region shapes or most other Visual Crossing layers, it is as easy as loading or copying a Shapefile from the Visual Crossing metadata file system.

The primary rule of thumb is to always copy and move every file in the group as a single unit. Even if you have some 'extra' files in the set, it is a good idea to always keep them together. Some contain optional yet important information that you may use as you become a more sophisticated GIS user.

Prerequisite for your Shapefile

As we discussed above, a Shapefile is a native Visual Crossing file format, and loading a Shapefile into Visual Crossing can be done by simply copying the Shapefile into the appropriate directory. However, before doing that, it is a good idea to check to make sure that your shapefile is in the projection expected for your mapping system. Typically, if you are using a standard background map such as OpenStreetMap or Google, your projection will be EPSG 3857. (For more details on this, see part one of this Shapefile series.)

One way to check your projection is to look at the PRJ file in the shapefile file set. The PRJ file is simply a text file. So you can open it in any text editor such as Notepad on Windows. (Technically speaking, the format of the PRJ file is known as Well-known Text format. You can use Google to learn more or just ask here for more details if you are curious.) Unfortunately, the contents of this file will vary quite a bit, and it can be rather difficult to easily determine the compatibility of a Shapefile. Even worse, adding newlines and other “whitespace” in the PRJ file is entirely optional and often omitted. So simply finding the right piece of the PRJ to read can be tricky. If you are lucky, you may see that ESPG 3857 value directly in the text. However, you are equally likely to see some piece of complex text that has a similar meaning but looks completely different.

A more pragmatic way to test your Shapefile is to load it into a stand-alone viewer tool such as the OpenJump tool that we discussed in Part 1. Then you can either add a background layer (OpenStreetMap, Google, etc.) if your tool permits or simply add a known-good Shapefile layer that matches the same part of the World as your data. You can find a US States layer in your standard Visual Crossing metadata under datasources\_default\USA\USA_states.shp. If your geography is more global, you can use the standard world countries file in datasources\_default\_INT\ _INT_Countries.shp. Once you have both layers loaded in the viewer tool, you can compare one overlaid onto the other. If your Shapefile data is in the right area of the map relative to a known good Shapefile, then it is likely projected properly and ready for use in Visual Crossing.

The good news is that Shapefiles that match the OpenStreetMap/Google standard are very common. So if you have a random Shapefile, especially if it is from a modern, commercial source, there is a good chance that it is already projected properly for use in Visual Crossing. Most publicly available shapefiles are ready to go directly. The publicly-available files that don’t match this standard typically come from non-US governments, local governments, or from older sources that were produced before Internet mapping became common.

Loading a Shapefile into the Visual Crossing Metadata

One you have verified that your Shapefile is ready to go, the next step is to copy it into the Visual Crossing metadata. This is as easy as copying files on the filesystem. Simply go to the root of your Visual Crossing metadata on your server machine and navigate into the datasources directory. In there you will find the standard Shapefile layers that we ship with the Visual Crossing product as well as any custom Shapefiles that you have purchased or otherwise obtained. While you can copy your files directly into this directory, it is a good idea to make a subdirectory based on the source, theme, or owner of your files. This will help you keep them more organized. Visual Crossing will recurse into every subdirectory under the datasources folder looking for layers. So you can make the folder structure as descriptive as you desire.

The only critical bit to keep in mind for this step, as we discussed in the first installment of this Shapefile series, is to make sure that you copy all of files of your Shapefile together. Remember that these files consist of every file with the same root name as the dataset. For example, the a US States Shapefile may consist of USStates.SHP, USStates.DBF, USStates.SHX, USStates,PRJ, and several others as well. So make sure that you copy the entire set of files into the Visual Crossing metadata together. Maintaining this simple rule will avoid much confusion later.

Closing thoughts

In closing, now that you have the Shapefile loaded into Visual Crossing, the fun can begin. You can use the new layer for visualization, geo matching (in order to visualize business data), and geo analysis. In the next installment of this series, we’ll dive into Visual Crossing and see how we can make use of our new layer and its data.