Visual Crossing Blog

A Primer on Shapefiles

In a tool such as OpenJump you can move geographic features and also edit attribute data.

If you use mapping and GIS systems more than occasionally you will almost certainly run into the Shapefile data format. While most good mapping systems support various input file formats, the Shapefile format is one of the oldest and most ubiquitous. Visual Crossing is not an exception to this pattern and is totally capable of using shapefiles both as a source for geometries to represent business data as well as external information such as demographics, weather and more. So, if you are going to be working with Shapefiles, it is a good idea to understand a bit about them. In this article is the first in a series, we'll explore some of the fundamentals of the shapefile format.

A Shapefile is not just a file

This may sound odd, but the first thing to know about Shapefiles is that the data is not contained in a single file. Instead, a Shapefile is actually a group of files with the same root name and various extensions. Some extensions are required and must be part of the group while others are optional. Let’s consider a shapefile for US States with the root filename being US_States. The minimum required files for the set would be US_States.shp (the geometry data) and US_States.dbf (the attribute data). You typically also have US_States.shx (an index file) and US_States.prj (a description of the projection and datum). You may also have some other, less common extensions as well.

The primary rule of thumb is to always copy and move every file in the group as a single unit. Even if you have some 'extra' files in the set, it is a good idea to always keep them together. Some contain optional yet important information that you may use as you become a more sophisticated GIS user.

Projection and Datum

Just as visible maps can be rendered with different projections and in different ways, the geometry data in Shapefile format can be stored in different ways as well. Visual Crossing, like many mapping systems generally uses data in the same projection and datum as common basemaps such as OpenStreetMap and Google. This format is known as Spherical Mercator WGS84 (or sometimes 'Web Mercator'), and it also goes by the very tech-sounding code name 'EPSG 3857.'

One common way that you will notice when you have a badly projected shapefile is that the data will not display properly aligned with your basemap or other, standard layers. In some cases, the points or polygons may be in a different location on the map while in other cases the layer data simply will not appear in the map range at all.

Luckily, if you have the PRJ file in your Shapefile group you can easily convert your data to match the common standard. One easy, opensource tool to do this conversion is the oddly-named OGR2OGR. If you want more details on how to use OGR2OGR, a quick search in the Visual Crossing knowledgebase will yield a technote with all of the details on how to download, set up, and use this powerful tool. If you do find yourself with a mis-projected Shapefile, a tool such as OGR2OGR can adjust it for you in a matter of seconds. For this reason, there is little to fear when you encounter projection issues.

Editing a Shapefile

Editing a Shapefile is a task best left to experienced GIS users. However, if you find yourself wanting to make simple edits, there are various opensource and commercial tools for that task also. One popular opensource tool is named OpenJUMP. Using this tool, you can edit both the geometry information and the attribute information. So, for example, you can move a line in a polygon or shift a point to a different location. You can also edit the attributes such as changing the name of a city or even adding a new value, such as an abbreviation, for each geographic element in the layer.

One word of warning, however, is to never try to edit the pieces of a Shapefile group independently unless you are a real expert. A common mistake is to try to edit the .DBF file in a DBF editing tool such as Microsoft Access, OpenOffice, or an older version of Excel. While this will appear to work and produce a valid DBF output, you will often find that the resultant DBF no longer aligns perfectly with the SHP geometry data. This can produce both obvious and extremely subtle errors in your maps. If you are unlucky these errors can be very difficult to find. So, you should always use a wholistic Shapefile editing tool such as OpenJUMP or ESRI in order to avoid costly mistakes.

Closing thoughts

First in a series: This article is the first in a series of posts discussing Shapefiles, their use in GIS systems, and how specifically to use them within a Visual Crossing environment. Future topics include how to install custom Shapefile datasets in a Visual Crossing metadata and how to use those Shapefiles on your own maps. Is there some topic related to Shapefiles, Visual Crossing, or GIS in general about which you would like to learn more? If so, please post your questions, comments, and topic suggestions in the comments below, and we’ll be happy to have our GIS experts address them for you.