Geodatabases, Attributes, and Domains

Introduction

GIS at its very core is not just a set of geographic points, it is geographic data tied to other information. The analysis of this attribute data along with spatial data is what makes the core of GIS so applicable and useful as a tool. The goal of this assignment was to understand how data in the field is captured and implemented into a geodatabase. While this may seem like a straight forward process, having multiple people collecting data can be problematic and this may come from lack of standardization on many levels. The first thing to consider is how the data are going to be captured, and how people will be imputing values into the database. For example, do people record numbers as words, or by characters? Are directions put in as letters or words? While these discrepancies seem small, when it comes to implementing or analyzing data, these differences can cause incompatibility within tables and data sets. In order to eliminate these issues the constructor of the geodatabase must understand the data that will be collected and constrain the people in the field to record the data in a uniform way. This can be accomplished with domains, which determine the type of information can be recorded for each attribute, and how that information is to be recorded.  

Methods

In order to thoroughly understand why domains are important, not only were we to construct our own geodatabase and create domains for the attributes we would be collecting data for, we also went out into the field and attempted to capture GPS data for a micro-climate survey of the UWEC campus via ArcPad. The benefit of a mapping grade GPS is that not only can geographic points be captured but attribute data can be multiple users collecting data can be aggregated in a usable way in one geodatabase. But, as stated before, in order for that data to be usable the data inputs and entry must be standardized such a way that data is usable with out post processing. The attribute data that we would be collecting for the micro-climate survey are; group number, temperature, date, cardinal wind direction, azimuth wind direction, dew point, relative humidity, and a notes field. After creating a geodatabase and these attribute fields, we applied domains to the fields such that each field would be limited to a standardized input of our choosing (Fig 1).

Fig 1. Creating Domains in the geodatabase. 

After creating the geodatabase, we added a image from the Eau Claire County geodatabase of imagery of the area (Fig 2). It was very important to have a clear and concise file path structure such that all data would be organized and data coming in from the field would be in a easy to locate location (Fig 3)

Fig 2. The survey area of the micro-climate survey, the UWEC grounds, outline in red.

Fig 3. The filepaths for the micro-climate survey data, related maps, image files, and geodatabase. 

The next step was to prepare that data for deployment by turning on the ArcPad Data Manager, and transferring the geodatabse and the image file to the Juno unit for GPS data collection (Fig 4).




Fig 4. Setting up ArcMap to deploy geodatabase, and thematic layers to Juno Device via ArcPad Data Manger.

Then we were to go out into the campus mall and capture data with a Juno handheld device which had ArcPad installed, and use a kestrel to capture information on wind direction, temperature, dew point and humidity (Fig 5)

Fig 5. Kestrel and Juno device for data capturing. Juno displays the Image file of campus that was loaded during the data deployment step.

Results/discussion

The exact study area was the University of Wisconsin Eau Claire campus mall, in Eau Claire Wisconsin. The weather outside was overcast and rainy, with a slight wind, and the rain was slowly turning to snow. After we arrived at the campus mall we had to turn on the location ability of the Juno device so that our positions could be recorded and we received a short demo from Dr. Hupy on the use of the Juno device (Fig 6).

Fig 6. Dr Hupy showing us how to use the Juno.

While the data collection went smoothly for most people, I ran into an early problem. When attempting to capture GPS data, I of course had to fill out the attribute fields that went along with each position. Everything was going well until I hit the notes field, which became auto filled with a <Null> value and would not let me capture a data unless the field had information in it, which the field would not let me input. This was a problem because, not being able to input any information as the notes field being empty would not allow data to be collected. After being generally confused about this and trying to figure it out for a few minutes, Dr. Hupy told me that it was because I probably had applied a domain to my notes field, which made it unable to record any information (Fig 7). Due to the fact that this was occurring I was unable to capture any GPS data and therefore unable to upload any data points to my map. For this instance not capturing data was okay, due to the fact that this was a trial run to understand how hard it is to create a geodatabase and implement domains to deploy in the field. In our next exercise we will be doing an actual microclimate survey, where we collect data in a pre-constructed geodatabase , with domains that have already been set up.

Fig 7. Error occurring as I had applied a domain to the notes filed, which did not allow for any entry.

Conclusion

I know now how hard it can be to capture all of the attribute data that makes GIS so useful, and why it is so important to make sure that the captured attribute data is normalized. With out capturing attribute data, GIS would just be a series of GPS data points with no functionality and would just be something to create maps with no information other useful information. But just capturing attributed data is not enough, in order to make sure that the data is useful, it has to able to be stored, deployed, and recorded in a way that anyone can use or record the information. For this to be possible the geodatabse that the information is recorded into or deployed from has to be set up in a usable and straight forward fashion that makes adding or using the data easy, and non-complicated for anyone to use. 

Navigational Map Construction

Field Activity #5: Development of a Field Navigation Map and Learning distance/bearing Navigation.

Introduction

This weeks our task was to prepare a set of maps that will be used in a future activity in navigation. Two things are required to navigate, a set of tools that allows the user to determine their position, and some type of projection in which that position can be referenced. In our case we will be using a compass and a set of maps of our own creation. We will be using these tools to navigate a preset course around the grounds and woods of the Priory at the University of Wisconsin Eau Claire.  

Methods

In addition to knowing the direction in which we are going, we will also need to know how far we are traveling from point to point. The method we will be using to determine this distance will be a pace count, knowing how many steps a person takes given a set distance. In this case our pace count was determined over a span of 100 meters. We laid out two 50 meter tape measures, end to end, on the sidewalk and then counting the number of steps taken with just the right foot over that distance. After determining our individual pace counts, we were given a geodatabase with information and various map layers and were told to construct a map that we thought would be the best to use in our future navigational activity. 

The layers that we had to choose from were aerial imagery, a section of the USGS quadrangle, black and white imagery, and 2 ft contour lines imposed over the area of interest. What I decided to use were the layers of aerial imagery and contour lines. 

The base of any map is the coordinate system and the projection, a developable surface tied down with a series of data points which allow a coordinate system to be tied to the earth in a flat representation of the Geoid and ellipsoid model that is the Earth.  In this case it is Nad83_WTM, and the projection is Universal transverse Mercator (UTM). UTM is designed so that that the world is divided east to west into 60 zones, with each zone being 6 degrees of longitude and distortion is minimized in each zone.

The UTM Coordinate System Zones of North America.

Eau Claire County is located in Zone 15. Then coordinates inside each zone are recorded using decimal degrees, a location of angular distance from the center of the Earth. 

When deciding on what layers to include in the map I would be using, the contour lines at two foot intervals seemed to make the map too cluttered to effectively use (Fig 1), so I decided to remove every other contour, effectively creating 4 foot contour lines. In order to accomplish this I had to select every other line first, which actually proved to be problematic. The contour lines were so dense in some places that it was hard to tell them apart, which would prove even more challenging when trying to navigate.

Eventually I found a command with the "select attribute" feature in the contour line table, on ESRI's website which allowed me to select every other contour line based on the object ID field with the script, MOD("OBJECTID",2) = 0 (Fig 2). This script allowed me to select object ids in intervals of two, but i also attempted to select various intervals and see what the map looked like then. I then removed the contour lines and began building my first map.

Fig 2. Every other contour line selected. 

For the second Navigational Map, I decided that packing more information on another map would probably not help when navigating in the field, as it may become confusing attempting to decipher information when using two maps to establish a reference point. So I decided that the second map should be more of a reference point of the big picture area of interest, so I removed all theme layers, and added only the Ariel photos with a reference grid in Decimal degrees (Fig 4).



Results/Discussion

The finished maps, (Fig 3,4) with grids imposed over the top of the maps for a reference point via navigation. Fig 3 shows the Contour lines navigational map with a measure grid. Each grid square represents both 50 meters in the X and Y direction. Pace count, directional arrow, scale, real world scale, and sources included on the right hand informational column. 

Fig 3. Finished Navigational map with all map elements and grid. 

Fig 4 shows the Contour lines navigational map with a graticule grid. Each grid square represents 0 degrees, 0 Minutes and 1 second in decimal degrees for the X and Y direction. Pace count, directional arrow, scale, real world scale, and sources included on the right hand informational column. 

Fig 4 Finished Aerial photograph reference map with grid.

Conclusion

While the second both maps seem to be a little spars in information, I believe that it will serve to be of greater benefit to have less information in the field when using either of these maps, as the maps wont distract the user with unnecessary information. The only thing that I would change after doing the project would be to make the grid lines to be more of a contrasting color such that they would be easier to detect. When the maps are in the size they were meant to be (11"x17") the do show up better than what is represented here.

Data Interpolation in ArcScene

Introduction

            In our previous lab we conducted a scale-able field survey to understand the basics of how field surveys are conducted. We constructed the field survey in the garden planter boxes in the courtyard of Phillips Science Hall on the lower campus of the University of Wisconsin Eau Claire. In the planter boxes we first sectioned off a 1 meter square area, and leveled the snow in the area with the top of the box, then we constructed a land scape and added the following features in snow: a ridge, hill, depression, valley and a plain. In the last survey we constructed a grid out of push pins and string in the 1 meter square area, by overlapping the string we created a grid pattern, made up of 5cm² (20x20), with a total sample of 400 possible data points. We then sampled each square for a “Z” measurement made up of a positive of negative elevation from the level surface of the snow which we made “sea level” with a 0 value. Once the data was captured, we put the grid and elevation values into excel and examined the data.

(For photos and further explanation, please see my first post Ex 1: Sandbox Survey)

Now that we have the data from our survey in excel, in today’s lab we will be normalizing our data, or taking the data that we have and factoring out the true size of the data to transform the data in to a standard measure, which we can work with. We will do this by turning our data points into a list of XYZ coordinates in excel, and then using these coordinates to map a continuous surface. Then we would use various interpolation methods to create an estimation of surface values at unsampled points based on known surface values of surrounding points. By doing this in ArcScene we can visualize our data points in three dimensions and see how accurate our surveying method was.

Methods

The first thing we did was, to take the data that we had entered into excel the first time and correct it such that it could be loaded into ArcMap as XYZ coordinates.

Fig 1: Table of entered data, fresh from the first survey. While this was an easy way to examine our data, it could not be used in ArcMap.

Fig 2: Corrected XYZ data which can be used in ArcMap

Once our data was loaded into ArcMap as a points feature class, we created a shapefile, and then turned the shape file into a layer file and added that file to a geodatabase. From there the file could be opened in ArcScene and a 3 dimensional picture of our data points took shape.



Fig 3. Data points in ArcMap.

Now that we had our data in ArcMap, we had to decided which method of data interpolation would be best to visually convey our data with the most accuracy. Data interpolation will predict the value of cells in a raster that may have missing our limited numbers of data points. The following data interpolation methods have various advantages and disadvantages. Below is a picture of survey data before any interpolation methods were applied.

Fig 4. Data points in ArcScene with no interpolation.

IDW Interpolation: Stands for Inverse Distance Weighted. This method of interpolation uses a method of predicating cell values based on the average of the values of the data in the neighboring cells. The closer a point is to the center of the cell being estimated, the more influence, or weight, it has in the averaging process. 
The inverse distance weights do have limitations, as IDW is an average, it cannot be used to generate surfaces in areas that have lower values than the lowest input that is in the data, or the highest value that is in the data. Therefore if a data point at high or low elevation is not directly captured then IDW will not be able to model those points accurately. Therefore if a ridge or value is not captured in the surveying process it can not be generated by this interpolation process. This means that in order for this model to accurately represent data of various elevations, a dense sampling technique is needed. (ArcGIS 10.3.1 Help)

Natural Neighbors:  This interpolation method finds the closest subset of samples to a particular point and then applies weights to them based on proportionate areas to find a value of a point that is not represented. Again this method is unable to create data points of peaks, ridges, pits or valleys that have not been surveyed.(ArcGIS 10.3.1 Help)

Kriging Interpolation: is from a second family of interpolation methods which is based on statistical models that include autocorrelation such that, the statistical relationships among the measured points. Because of this, geostatistical techniques not only have the capability of producing a prediction surface but also provide some measure of the certainty or accuracy of the predictions. (ArcGIS 10.3.1 Help)

Spline Interpolation: This method uses a mathematical function that minimizes overall surface curvature, resulting in a smooth surface that passes exactly through the input points. Spline operates on two premises. 1) The surface must pass exactly through the data points. The surface must have minimum curvature. 2) The cumulative sum of the squares of the second derivative terms of the surface taken over each point on the surface must be a minimum.

TIN Interpolation: The purpose of the Raster To TIN tool is to create a triangulated irregular network (TIN) whose surface does not deviate from the input raster by more than a specified Z tolerance. The negatives of a TIN interpolation are that the continuous surface that is generated will not have any smooth or rounded surfaces, meaning that while the the TIN network will show peaks (to a point) it will not show an accurate depiction of the landscape, but rather specific surveyed points. 

Once the data was in ArcScene and we knew a little about the advantages and disadvantages of each interpolation method, we tested each interpolation method out in order to decide which most accurately represented our constructed terrain. One important correction that we had to make was to adjust for the drastic highs and lows that the 3D interpolation models suggested for our peaks and valleys. (see fig 5 Below). While this is represented below in a TIN network, this was done for all interpolation methods.

Fig 5. A Tin interpolation of our data with uncorrected peaks and ridges.

In order to correct for this, the layer properties of the surface was normalized such that a factor of .25 was used to convert every elevation value into a more accurate surface. (see Fig 6 below).

Fig 6. A conversion factor of .25 was used to scale the proportions of the elevation into a more accurate representation of the true constructed terrain.

When contrasting the 2D map with its 3D counterpart, the best comparison came when both representations were in the same orientation in ArcScene as in ArcMap. Having determined the orientation, a scale bar was created to reflect the true size of the 3D model in comparison to the 2D model, as showing the 3D surfaces changes the perception of size in the model and may disproportionally make the 3D model appear larger than its 2D counterpart. 

Results/Discussion

Each method of Interpolation is shown below in a map such that their is a juxtaposition of the 2D and 3D models for each data set. Now the question becomes how well does the interpolation model match our survey and the terrain we surveyed. 

Fig 7. A reminder of the created terrain, 

Kriging Interpolation:

Fig 8. Kriging Interpolation method of surveyed terrain

The Kriging interpolation method averages out a lot of the surface terrain, and rounds it out to a great extent. The first issue that I would point out is the top of the mountain in the upper left hand corner of the 3D model (fig 8), the surface of this feature has been rounded off and dose not show any of the sharp edges that the feature truly had. Similarly, this theme is repeated in the valleys, where the sides of the valley did not appear as steep nor did the ultimate depth of the valleys appear to be as extreme as we had surveyed them. The ability of the Kriging interpolation to average the terrain may work better at a larger scale or on terrain that has more gradual inclines, but it does not match our data very well.

Spline Interpolation

Fig 9. Spline Interpolation

The best method of interpolation has to be between the spline (fig 9, above) method and natural neighbors (fig 10, below). While both methods did not capture the details of the hill in its entirety, they did the best job of capturing the ridge, the valley, and the depression. While looking at the 3D models of each method they do both look quite similar, however when looking at the 2D model, Natural Neighbors was able to show more details on average depth of the valley, which appears to be slightly different in the Natural Neighbors 3D representation (fig 10). While the ridge was not as represented in the 2D model of Natural Neighbors, comparison to the spline interpolation method dose not appear to have a drastic difference in the 3D model between the two. With this in mind I would have to say that Natural Neighbors was the best in representing our survey, even though it did have some drawbacks.

Natural Neighbors

Fig 10. Natural Neighbors

IDW Interpolation

Fig 11. IDW Interpolation

IDW Interpolation was a very accurate method in representing our hill, again in the upper right hand corner (fig 11). IDW's ability to layer the elevation of the hill and contrast each layer really serves to build up the hill in contrast to the plains upon which it sits. IDW does not do a very good job in representing the ridges on the far right of the map. While it does shoe that these ridges are higher than the surrounding area, it does not model them to have a distinct ridge or sharp edge at their peak which our model dose not necessarily portray drastically but it is more present than what is depicted here.

TIN Interpolation

Fig 12. TIN Interpolation

While TIN interpolation was probably the least accurate in terms of representing the natural curvature of the features that were created, this method was a personal favorite of mine. TIN networking really shows off actual elevation of the specific points that we surveyed and does a very good job of portraying the survey points relative to each other due to its category breakdown for elevation. For example the individual points that we captured on the top of the hill are captured in the 3D model to a more accurate extent as opposed to some of the other interpolation methods, as well as the peaks of the ridge on the right side of the map.  The negative of the TIN network are that the model does not do gradual changes or stepped survey points, it rather averages these points into a flat surface. This dose not leave the best relief of the features we created. While TIN does have negatives, TIN also happens to look like early video games and reminds me of my childhood. While I can see its uses for interpolation data I would not say that it did the best job of representing our survey in a 3D model. 

Revisit your survey (Results Part 2)

For the next part of our survey, we had to note which interpolation method matched our data and terrain the best and then reconstruct and resurvey those points that were lacking in an attempt to improve our model. While the best interpolation method was Natural Neighbors, as noted above, the model lacked accuracy when it came to the hill data, and as a general trend in all the models, none of the interpolation models did a fantastic job of capturing the valley.

On a very cold day ( a balmy 3 degrees), we remade our terrain in same planter box in the Phillips Science hall court yard in Eau Claire. We attempted to match our previous terrain as best as we could, but the snow did not compact in the same way as it had when we had constructed our first model. This translated into features that did not have stark ridges or edges, but rather crumbling terrain. 

For our second survey we wanted to capture data in greater detail so we constructed a double grid, such that our first grid of 20 x 20 with 5 square CM individual squares was the base of an identical grid now shifted 2.5 cm up on the Y axis which ran East to West in all of our maps (fig 15). The second grid was constructed using the same push pin technique with a different colored thread so that we could tell the two grids apart. We then focused surveying on the specific features, their edges, and elevations such that more sample points were created in places that had a none zero elevation. 

Fig 13. The construction of the double grid.

Fig 14. second survey with single grid. General terrain features were attempted to be captured and reconstructed from our first survey.

Our double grid did give us more survey points around the features we wanted to capture, and were able to capture double the survey points as in our first survey, however only expanding the Y axis of our survey created an elongated map and 3D model. So while our new Natural Neighbors interpolation (fig 16, below) has more accurate elevations, it is skewed to have a double length and singular width, and actually causes our 2nd survey models and maps to appear disproportionate. 

Fig 16.

Part 5: Summary/Conclusions

While the second survey did increase the number of data points that we were able to gather more data, it did not appear to have increased our modeling accuracy. As to whether this was a result of our survey techniques or grid creation I am not sure, but I would predict that increasing the number of data points in a specific area would increase the resolution of the map and model and therefore increase the accuracy of the Interpolation, no matter which method was chosen. I do not think that such a detailed ground survey would always be realistic given that there would be a great deal of data points and such a vast distance in which people would have to move in order to capture those points. There would be to many logistical issues to cover to capture that many moving pieces. 

Interpolation can be used to infer data in many instances such as rainfall, temperature, chemical dispersion, or other spatially-based phenomena.