GEMs Software Review

This week I will be providing a review of the GEMs software used to develop maps from the data collected in the previous weeks with the corresponding hardware on the multi rotor.

Overview of Software:

The GEo-location and Mosaicing software or GEMs is a used in the compilation of images gathered by the GEMs hardware mounted on a fixed wing or multirotor aerial photography platform. The hardware consists of a multispectral sensor that can collect images in color (RGB), near infrared (NIR) and several forms of normalized difference vegetation index or NDVI. This system was specifically designed to aid in agriculture since the main role of the NDVI imagery is to identify the health of crops and pinpoint any problem areas in a field such as malnourishment or bug infestations. When mounting the hardware to an aerial system it is important to point the antenna away from all magnetic objects or other antennas this will aid in preventing any extraneous signals from disrupting data collection. Adding a copper plate beneath the GEMs will help to act as a faraday cage which will essentially absorb or and eliminate any electronic interference.  It is also important that the hardware mounted facing downwards in a location were vibrations are minimal or dampened to improve image and data quality. As the mission is being flown, data is collected on a SanDisk Extreme 32GB storage device which is able to write data at 100MB/second. The GSD or ground sampling distance is a relationship between the area covered in a pixel at a certain height. For the GEMs the GSD at 200 feet is 2.5 cm and at 400 feet is 5.1, which shows that the relationship between sample height and GSD are near linear. Pixels can also be geo-located to within 1.5 centimeters with field markers at best and without markers between 3 and 5 meters. The sensors are programmed to enhance a variety of parameters such as rate of coverage, field of view, platform altitude, platform velocity, image overlap, percent smear, exposure time and GSDs. These automatic adjustments are invaluable as they save a lot of time in the image collection since you won’t have to worry about doing calculations in the field to measure out these values. At the conclusion of a mission using GEMs, the GPS coordinates are automatically associated with an image for each of the three image types and an orthomosaic is created. The GEMs software allows for two different types of processing when creating a mosaic, which is the compilation of all the images formed into one geo-located image. These processing types are fast and fine mosaicing, where a fast mosaic will be a quickly processed image based on the navigation data corresponding to the individual images and fine will process the image more in depth ensuring proper alignment of the images and creating an overall better quality mosaic. 

GEMs hardware can be used with different software (such as Mission Planner as was used in our class), and if that is the case the following information can be used to ensure successful data collection.

• Image sensor resolution: 1280 x 960 pixels 
• Sensor dimensions (active area): 4.8 x 3.6 mm 
• Pixel size: 3.75 x 3.75 μm 
• Horizontal Field of View: 34.622 deg 
• Vertical Field of View: 26.314 deg 
• Focal length: 7.70 mm 

Using the Software:

To analyse the data previously collected using the GEMs hardware, the imagery data was uploaded onto a computer interface and a file was created to isolate the data for a particular mission. When in the folder, the data is automatically named to represent the time and date that it was collected. This is a useful feature so you can keep track of your images. The data was then able to be uploaded to the GEMs software where the fisrt step is to initialize the NDVI data. This step besides the part where you click the 'Run NDVI initialization' button is automatic and a loading bar will pop up to show you the status of this initialization. Following that, the data was ready to be sent to a program called ArcMap where the data can be visualized and a map can be created. The files that we want to use in ArcMap are called .tiff files which are created when the NDVI data is initialized with the GEMs software. TIFF is a file format that is used to store raster graphics which contain data relating to the color of a pixel. This is important for NDVI images because the color of a pixel determines the health of a that area of vegetation which is the whole purpose of collecting NDVI imagery.

Speaking of pixel colors, here is how pixel color relates to image type:

• RGB Image: Each pixel contains a certain percentage of Red, Blue or Green which related to color of the image as it pertains to the visible light spectrum
• NIR (Mono) Image: This is imagery taken near infrared, this imagery is interpreted in such a way so that you can see it since we biologically cannot see in infrared. 
• NDVI FC1:  Here, the darker orange the pixel, the healthier the vegetation, bad vegetation is dark blue to black in color
• NDVI FC2: This imagery displays healthy vegetation as green and bad vegetation as red.  In this situation, green is good and red is 'dead' an easy scale to visualize. 
• NDVI Mono: Here the more white a pixel, the healthier it is. A black pixel would indicate very unhealthy plant life.

Another form of TIFF file is a geoTIFF which allows georeferenced data to be embedded withing the TIFF file. You can probably understand why this would be important if you want to use your imagery for mapping. In ArcMap these geoTIFFs are used to place your image on an underlaying base map of the world, a feat almost impossible if you don't have any geographical reference points. Other image types such as a .jpeg are not automatically geotagged and therefore whould cause problem if you wanted to create a map with them. One thing the GEMs software will not automatically do is update the metadata associated with your imagery. To combat this it is important to keep a good record with you metadata so that others viewing your data will have an easier time interpreting it and understanding where it came from.

Summary:

With this being my first time analyzing data using the GEM software, it was a bit intimidating at first. Luckily a lot of the data analysis was done automatically making the process a little more streamlined and easier for a first timer like myself. The resulting processed imagery was good, but I am unsure whether it was the software or just the images, but in the NDVI imagery you can clearly see distortions that make the imagery at those points less than ideal. Given, it does a lot better of a job than I could do manually that is for sure. This equipment does not come cheap however and is near $8,000 to obtain the hardware alone, which I would consider a downfall for the technology for those who want to use GEM for individual use and don't have a large budget. Overall I would rate the GEMs as good, but there is still room for improvement. 

Field Activity 5: Gathering Oblique Images with UAS for 3D model Construction

Introduction:

All of the aerial photography that we have gathered thus far in the class has been taken with the camera pointing directly down at the ground, or nadir. This style of photography is good for producing two dimensional imagery, but doesn't have the depth needed to produce a three dimensional model.  To produce a three dimensional model, the camera must be shifted from nadir to oblique, where oblique camera angle will range between 0 and 90 degrees allowing us to gather images with more depth making it easier to develop a three dimensional model.

Study Area:

For this study, our group traveled to the Eau Claire Soccer Park again in Eau Claire, Wisconsin. The soccer park was a good location because at it's center was a concession stand which was the perfect size to gather enough imagery to create a 3D model with the UAS system that we had. The weather on the day of the study was mostly clear and sunny with very light winds to the south.

Figure 1: Concessions building that was surveyed outlined in red.
North is indicated by red arrow in the bottom right hand corner

Methods:

Figure 2: IRIS multicopter
photo courtesy of drnes.com

This study was done using two UAS systems; first the Iris and then the DJI Phantom. First we used the Iris which was programmed with mission planner to take images at certain intervals during the flight with a non-fisheye lens. The program for the image gathering was designed so that the Iris would be pointing it's camera at the building the entire time while it ascended upwards in a helical pattern from 15 to 26 meters. This program is also featured on the tablet format of mission planner as structure scan mode. Once it reached it's maximum altitude, the Iris then gathered images in a zigzag pattern to gather images of the roof structure. Following this, image gathering ended and the Iris landed.

Figure 3: DJI Phantom Drone
Picture courtesy of www.teamonerepair.com

The second scan of the building was done manually with the DJI Phantom drone. Everyone in the class took turns flying around the building and taking pictures with the camera mounted to the Phantom. The controller had a tablet mounted on it so that the operator could have a first person view of the image before capturing it by pressing a button on the controller. The angle of the camera could be controlled with a wheel on the controller but it mostly stayed at the same angle for the entirety of the scan. There wasn't a strict pattern that was followed when collecting this imagery so some people chose to take some images of the roof, while others to pictures around the perimeter. Images were collected until the Phantom had used two batteries.

Discussion:

This discussion will focus on the image collection, be expecting a future post on the processing of the imagery into a 3D model. Unlike the images that we were used to collecting with the Matrix, the images on the Iris were taken with a Go Pro and therefore were not geo-tagged. This can lead to challenges when trying to pinpoint the location of your 3D model on a software platform like Google Earth. Luckily, however there are other options. The first option you can use is to assign coordinates for your model in Google Earth. A second method you could use to aid in placing your model is a survey GPS. Finally, a program called Geosetter can utilize the telemetry log, match the way-points and time stamp data and infuse this information into the image to give it a set of coordinates. Needless to say, if you are collecting imagery from a device that doesn't geotag images, there are still many ways to do it after the collection.

In the pictures below notice the differences in how the oblique images give the concession building more depth than the nadir image that was taken in a previous study. Even if the roof heights and angles were known, it is also difficult to create a 3D model when you can only see the roof and nothing below it. With the oblique format you can see connections between the roof, walls and pillars along with other building features that are necessary in a detailed model.

Figure 5: Nadir imagery collected in a previous field activity
with the Matrix. Notice how difficult it is to see the angles
and height of the roof  structure and how this would be difficult
to derive a 3D model from

Figure 4: Oblique imagery collected with the Iris multicopter.
Notice how the oblique camera perspective allows for better
perception of depth which is essential to 3D imagery

Conclusion:

This field activity focused on the collection of oblique imagery to be used in the construction of a three dimensional model of a concessions building. Previous imagery collected in the nadir format, though useful for other means is not useful for producing a 3D model as is shown above. With oblique imagery you can image the faces of the building and capture more of the intricate surface details that would be useful in the creation of a 3D model.

Field Activity #4: Gathering Ground Control Points (GCPs) using various Global Positioning System (GPS) Devices.

Introduction:

In this field activity, our class experimented with various methods of collection ground control from the very accurate to the very approximate. Ground control points, also known as GCPs, are used to improve the quality of aerial imagery and data collection if gathered correctly. At their most accurate, you can measure out the altitude, latitude and longitude within millimeters. Accuracy is important when it comes down to collecting survey grade data which some companies may need when require geospatial data to be collected at high temporal frequency to monitor a development plot in the mining or agriculture industry for example over time.

Study Area:

This weeks field activity took place near the South Middle School/Southside Community Gardens of Eau Claire, Wisconsin where just south is a large open area with walking paths and a few small bodies of water. This location was chosen due to it's varying landscape and to provide a different location other than a soccer field complex. The study area fell inside and included the gravel walking path that surrounded the northern most body of water, which wasn't visible from ground level due to it being surrounded in tall grasses and weeds. Ground control points were placed along the path around this body of water and with three of the six being placed inside the path in the grass. The weather was clear with a considerable amount of wind to the West.

Figure 1: Image of study area from google images. Area studied is
 outlined in red and North direction indicated with red arrow 
in bottom right corner. 

Methods:

Six ground control points were placed around the study area. Each of the GCPs were made from tarp-like material with holes at each corner for nailing the GCP into the ground. Each side of the square GCPs was the base of a white or black triangle, whose points met at the center to form what could be described as intersecting black and white hourglasses. When placing the GCPs it is key to place them within the borders of the study area where they are visible from above, that way they can be seen and will not be warped by being near the edge of a image. The first GCP was placed on the gravel path just south east of the end of Hester St.

Figure 2: Nadir image taken with Matrix UAS above
the first ground control point.

The second GCP was placed South of the first in the tall grass approximately half way down the path before the first turn. The third GCP was placed South of the second at the first Eastward turn of the path. Approximately halfway up the Northeastern pointing path there was a thin gravel path that extended towards the small body of water, at the edge of this path the fourth GCP was placed. At the end of the Northeastern path where the direction changes to go Northwest, the fifth GCP was placed and finally halfway up that path to the Northwest, the sixth GCP was placed in the tall grass on the inside of the path. While placing the GCPs the altitude, latitude and longitude were taken with the Dual Frequency Survey Grade GPS which is a high precision unit that would act as a comparison the other GPS units used.

Figure 3: Dual Frequency Survey Grade GPS
on mounted system 2m above the ground at GCP 5

Figure 4: Topcon display connected to
the Dual Frequency Survey Grade GPS
showing the reading at GCP 5

Figure 5: Bad Elf GNSS Surveyor
(yellow) used in data collection

The class was then broken up into groups and each group collected data at the GCP for a different type of GPS unit. Each device was placed at the center of the GCP (Where the triangle points met) and a data point was collected. This was done at all six GCPs. My group collected data using the Bad Elf GNSS Surveyor GPS, a unit that should have accuracy to within one meter and tens of thousands of dollars cheaper than the first unit used. A tablet app was used to gather data with this device. The next group collected data on a tablet device with the even cheaper Bad Elf GPS which was not survey grade. Other groups collected data with the Garmin GPS and collected geotagged images with a smart phone. After these data points were collected, a mission was flown using the Matrix quad-rotor UAS to determine how accurate the sensors were and how they related to the GCP markers. After all experimentation was done, the temporary GCPs were collected.

Results and Discussions:


Below are some of the data sets collected by the the Topcon Survey GPS and Bad Elf GNSS GPS. Notice the precision in latitude and longitude of the 12 digit Topcon compared to the 8 digit Bad Elf.

Topcon Survey GPS

GCP:  Latitude,            Longitude,       Altitude

1:  44.7773757474,  -91.4729374398,  267.839
2:  44.7767401655,  -91.4729385628,  267.670
3:  44.7758957110,  -91.4728733066,  267.487
4:  44.7763877743,  -91.4721470321,  267.580
5:  44.7769016214,  -91.4711962560,  269.177
6:  44.7773686301,  -91.4723883271,  267.341

Bad Elf GNSS Surveyor GPS

GCP:  Latitude,  Longitude,  Altitude

1:  44.777397,  -91.472922,  267.839

2:  44.776792,  -91.472942,  267.670

3:  44.775911,  -91.472892,  267.487

4:  44.776406,  -91.472128,  267.580

5:  44.776922,  -91.471192,  269.177

6:  44.777378,  -91.472386,  267.341

Figure 6: Study area with overlayed GPS location as recorded by the various units described on the key in the bottom right-hand corner for purposes of analysis of accuracy the XYZtopconsurveygps should be used as the most precise coordinates.

One of the main goals of this activity was to experiment with collecting GCPs with various GPS devices and determine their accuracy. If you refer to figure 6 above, you can see that the different GPS devices were indeed varying in their capability to provide accurate coordinates. It was expected that the most expensive and precise Topcon would outperform the rest in terms of accuracy and that appears to hold true. It was also expected that as you downsize in price range down to the collector app of an iPhone that the quality of data would decrease. Based on figure 6 we can see this theory is only somewhat true. Based on the image we can see that the iPhone does produce some obscure points around GCPs 5 and 6 on the right of the image, but the one that produces the most obscure results is the Bad Elf pro collector which is a surprising result.

As you can imagine the cheaper alternatives to the Topcon were easy to used and to collect data from with minimal set up at each GCP. All the user had to do is place the device approximately near the center of the GCP and click a button to record the point, however this simplicity shows in the accuracy. If you want to get precise data for high frequency collections to observe changes it is necessary to use precise equipment. Though it was time consuming to stabilize and re-stabilize the Topcon at each and every GCP it did provide accurate measurements that can be analysed over many surveys to be used in a commercial regard.

If you want to gain accurate and useful data it is important to have enough GCPs. It is key that you have no less than 3 GCPs for you study area. The number of GCPs also depends on the terrain. The more variance in elevation, the more GCPs you will need, likewise, the more monotone the study area, the more GCPs you will need to help stitch the images together. Obviously the more GCPs you have the more time consuming the process of collecting accurate data will be, however, your data will be more useful if you do this.   

Conclusion:

In this field activity, various types of GPS units were used to collect data at six different GCP locations. It was shown via the Topcon that the more time and expense it took to gather a point, the more accurate the data you will receive. However, cheaper survey grade devices had some accuracy, and could be used to collect data over a single study. The GCPs we used were only temporary, and therefore useful for a short term study like ours, but for in a commercial study our methods would not have been accurate and we would want to elect to use more permanent GCPs.