Field Activity #3: Using a multi-rotor to gather imagery/Using mission planning software.

Introduction

This week, we utilized the information that we learned in last week's field activity and flew automated missions using a multi-rotor UAS. This field activity helped us gain more experience planning missions with the Mission Planner software, engaging in pre-flight checks before every mission, updating flight logs, and staying engaged at a certain position (pilot in command, pilot at the controls and spotter) while the mission is taking place.

Study Area

For this field activity we chose to return to the Eau Claire Soccer Park in Eau Claire, Wisconsin. This complex provides us with spacious field with a minimum public presence during the late afternoon/early evening before soccer games begin. A wide open field with minimal people is important when flying multi-rotor UAS missions especially in the event of a malfunction or crash it is important no one is injured. Our survey mission was flown over a concession/restroom building that is located at the center of the park. The skies were overcast with light winds from the South.

Figure 1: Study area for this field activity. The mission was carried out in approximately same region as is outlined by the red box and North is indicated by the red arrow in the bottom right hand corner.

Methods

Upon arriving at the study area, the computer containing Mission Planner was booted up and the modem was attached. The modem was then attached to the Wonder Pole and raised into the air to its full length. For these missions we used the Matrix multi-rotor UAS. The Matrix was removed from it's container and assembled. The mission plan was created, a flight log was started and the pre-flight check was done. After the pre-flight checks, the mission was run. Each mission took about 6-7 minutes to complete and the data was saved to the sensor for later review. Two additional and identical runs were made with a Cannon SX260 camera with CDHK installed, the first with an RGB camera and the second with a near IR camera. A timer was set on both cameras to specify a time interval for image gathering, but could only be an estimate since the cameras were not linked to the Matrix hardware. After these missions were run, a post flight check was completed and the base station was dissembled.

Figure 2: Matrix multi rotor UAS

Results and Discussion

Three image gathering methods were used in our survey. The first method used was gathering image data from the sensor that was already mounted on the Matrix multi-rotor. This sensor was programmed to take images based on the mission planner software which takes into account the field of view allowing for acceptable amounts of overlap between successive pictures and passes. Two types of images were taken, one in full color RGB and one which was monochromatic. Below you can see that both Figure 3 and Figure 4 were products of the sensor imagery since both the images produced are nearly identical in position and orientation. The accuracy and overlap can be visualized in Figure 5 where I overlayed three photos to show photos one after another and in a separate pass. These images were so easy to overlap that I was able to do it in a Word document. This attests to the UAS's ability to maintain a very consistent altitude as described by the programming.

Figure 3: RGB photo capture from sensor mounted on the Matrix

Figure 4: Monochromatic photo capture from sensor mounted on Matrix

Figure 5: An overlay photo showing the overlap of the imagery collected by the sensor of the Matix. The blue box depicts the original picture depicted in figure 3, the red box depicts the next picture taken by the sensor and the green box  at the bottom shows the overlap of the photos taken in the next row pass.

The photos taken with the Canon SX260 cameras were not as accurate as the sensor imagery, but this is because they ran on their own platform instead of being programmed with the mision planning software. Since we had to use a simple "photo per time interval" approach, the overlap here was not as good. However, we were still able to gather some imagery as seen below in figures 6 and 7 with the near infrared and RGB cameras respectively.

Figure 6: Near IR imagery from Cannon SX260 camera
mounted on Matrix multi-rotor in place of first sensor

Figure 7: RGB imagery from Cannon SX260 camera 

mounted on Matrix multi-rotor in place of first sensor

Conclusion

This field activity allowed us to use a hands on approach to mission planning and carrying out a mission while reinforcing the concepts of maintaining a flight log and carrying out a pre-flight check. The RGB and monochromatic imagery that we collected with the sensor hardware connected with the software of the Matrix multi rotor turned out to be better than the camera imagery being taken from its own,separate platform. The imagery from the sensor apparatus was also able to show us just how accurate the altitude of the UAS is when it is running autonomously.

Field Activity #2: Using mission planning software and multi-rotor preflight checks

Introduction

For our second field activity, we learned how to use and develop surveying missions to be programmed to a multi-rotor using mission planning software, as well as running through the preparation for a multi-rotor flight. Mission planning and preflight checks can make or break a survey and therefore proficiency and thoroughness in both of these aspects is integral in collecting useful data and preventing the destruction of expensive equipment. This activity was carried out in a computer lab in Phillips Hall and the preflight check was done between Phillips Hall and Davies Center.

Mission Planning Software Methods

Figure 1: Mission Planner logo

The first step in flying a programmed mission is using the necessary software to design and plan said mission. This can be done with a program suitably called Mission Planner. When you enter Mission Planner, you are prompted with a world map. From here you can zoom in on the location in which you want to conduct your mission and begin planning it. Before planning a mission it is important that your mission area will not interfere with any no fly zones or other hazards like hospital helicopter routes. Airport no fly zone radiuses are depicted as purple circles on Mission Planner and have to be avoided. Once you find your area to conduct a mission, begin planning using the tools they provide. A simple point to point mission can be created by clicking on the map and selecting waypoints. Each waypoint can be customized to a certain altitude, speed in which it is reached, whether or not it is a take off or landing point and more. All this information from the actual mission will be stored in the telemetry log which will show what the UAS did in reality. waypoints can also be automatic using the drop box and creating various shapes like circles or boxes.

Figure 2: Mission Planner interface using a waypoint circle. Notice on the bottom left the ability to edit each waypoint

If you are using mission planning for surveying you can click <Auto WP to survey> and when you create a box on the map, an automatic zigzag pattern will appear. There are a variety of options to alter this pattern which will appear on the right side of the program. You have the ability to select your aircraft and camera, adjust speed, altitude and these adjustments will display on the mission. The program takes into account field of view and therefore if you do a mission at a high altitude, you will perform less zigzags and the mission will go faster. The opposite is true for a low altitude mission. The lower the altitude, the smaller the field of view and in order to get proper overlap for surveying, more zigzags must be made, lengthening the mission. It is important to find a happy medium between these too so that your mission does not exceed the time your battery will last. Mission length is calculated at the bottom of the screen automatically making it easy to determine which way you need to adjust your flight parameters. Once you have a mission that you want to fly, you can save it to be later uploaded to a UAS.

Figure 3: Waypoint Box Survey interface. Notice the right hand side displays aircraft selection, speed, altitude and preferred flight time. Also notice camera selection ability and mission area editing. Below you can see estimated flight specs such as total distance, distance between lines, flight time, area covered, and photo interval suggestion. To the left notice enlarged stats

Flight Preparation Methods

After planning a mission, you and your team have to travel to your study area with your UAS of choice and prepare to fly. It is very important that every measure is taken to ensure a successful and safe flight and his can be accomplished by following a preflight check such as the one below. 

Figure 4: Preflight Checklist

Before the preflight check, it is important to assign roles to each team member to ensure all facets of the flight are accounted for and monitored. There three positions that need to be filled. The first is the individual who focuses on the computer known as the pilot in command (PIC). The PIC's job is to fill out the preflight check by communicating with the pilot at the controls and set up the modem so the computer can communicate with UAS. The modem is attached to the computer via a USB connection and then is attached to the top of a "Wonder Pole" which telescopes upwards to establish a stronger connection to the UAS when it is in flight in an attempt to avoid communication issues caused by extraneous signal noise nearby. The PIC will also upload the mission plan to the UAS and ensure each step is accounted for as to not send the UAS on back to its last mission. During a mission, the PIC will be focused solely on the computer ensuring proper signal strength and satellites are obtained

Figure 5: Base station set up with wonder pole.

The second position is the pilot at the controls (PAC). This individual is in charge of physically checking to make sure all of the components the PIC is listing off in relation to the UAS and transmitter in the preflight check are in good operating condition and turned on. During a mission, the PAC is always at the ready with the transmitter to assume control of the UAS in instance where a mission becomes too dangerous (eg. oblivious citizens or other manned aerial vehicles present) or satellite signals are lost causing the UAS to lose GPS capabilities. This is why it is important that the PIC and PAC have good communication and remain focused even when a program is controlling the UAS.

Finally, a third position is the spotter. The spotter is in charge of assisting the PAC when needed with the preflight check. They can help with any last minute repairs or wherever they are needed. During a mission the spotter assists in keeping a visual on the UAS as you can never have too many people keeping an eye on the mission and making sure everything is running smoothly.

As you can see on the preflight check mentioned above, there are a lot of things to check and that is good. The more you take the time to check, the less likely you are to have a failed mission. I wont describe all of the checks since some are self explanatory, but there are some that have more too them that are good to know in order to have a successful mission. The first thing the PIC fills out is the Multicopter Log which asks for date, time, individuals acting as PIC, PAC and spotter, battery voltage and percent (both before and after), flite time, battery name, and the weather. It is important when filling out the battery information that you use voltage and not percent. While the may actually be half charged, once you plug it in it will display 100% since the computer considers that to be full charge. If you fly according to percent battery and the voltage is low, there is a good chance your UAS will lose charge and crash. It is also a good idea to check your computer battery to ensure that it will be functional throughout the whole operation.

Figure 6: Multirotor (Matrix) used in preflight check

Upon completing the multicopter log, the PIC then can begin the preflight check. This process involves the PIC yelling out the checklist to the PAC and the PAC responds with "good" or "okay" once the item in question is functional. The first few checklist items are to ensure that multirotor is structurally sound and connected. This means checking to see that the legs, frame, wires, battery and propellers are free of cracks, loose screws, dirt, worn down areas and anything else that might threaten the stability of the craft. It is important the check to make sure the battery is not only secure, but also in balance with the rest of the multirotor. An unbalanced battery can cause rotors to work too hard to level out the craft draining the battery quicker or cause it to not function properly all together.  The next part of the check ensures that the wireless communications are functioning and connected. This includes making sure Antennae are secure and out of the path of the propellers, connecting the sensor, turning the transmitter (TX) on and powering up the UAS. Once this is done the PIC makes sure the modem is connected, and that the base station is connected to the UAS. This is the computer in conjunction with the modem and wonder pole. Next the mission planning software is accessed and is connected to the UAS via a detected number (eg. 50600). A loading bar will appear and afterwards the platform will show up on a map via GPS. At this point you can look at the number of satellites currently connected to the UAS and make sure that this number never falls below 6. The more satellites connected, the better the UAS can position itself in three dimensional space. Lose that capability and your mission could fail. Another helpful number displayed on the mission software is the h drop. This number displays the horizontal differential position a number you want to be low.
Continuing on with the preflight checklist, the battery voltage and percent need to be documented again, and once the mission is uploaded, all the waypoints are written and the sensors are on and ready the UAS is ready to enter the takeoff sequence.

Before taking off it is important to clear the area of all spectators. THIS IS VERY IMPORTANT. Though UASs are cool, they can be dangerous if handled incorrectly or if others are acting carelessly around them. To prevent unneeded injuries it is important to make sure the area around the UAS is cleared. Before activating the UAS it is important that the TX throttle is down so that when it is connected with the UAS, it doesn't cause it to throttle up unexpectedly. The kill switch on the TX is then deactivated and the TX is armed. The multirotor is then sent up into a loitering mode and satellite connectivity is checked again making sure that number does not fall below 6. The mission is then free to begin. Upon completion of the mission, the UAS will return and the base station, battery and TX can be disconnected. The sensor should be checked again too.

Discussion

There hasn't been any data collected from this activity yet, but it was interesting to have a hands on approach to creating a UAS mission and preparing and arming a UAS for flight. Learning about the necessary steps and precautions needed to be taken before operating a UAS was a surprisingly long, thorough and much more necessary process than I thought it would be before coming into this weeks class. All UAS operators of all backgrounds should follow a similar process in preflight checks to ensure the safety of themselves and those around them. If  accidents occur because of careless flying of UAS it could be detrimental to the viability of this technology for use in the public domain.


Conclusion

Mission planning and having a thorough preflight check are important components in any successful surveying mission. Any deficiencies in either can lead to failed missions or even worse, destroyed equipment or physical injuries. Anyone who wants to conduct UAS missions must be strict with following safety guidelines and have a good handle on the functionality of their UAS for their safety and for those around them.

Field Activity #1 - Image Gathering Fundamentals

Introduction

The goal of this activity was to explore the basics of image gathering using a simple helium balloon and picavet rig apparatus. Originally this activity was to be completed using a large kite, however the activity began with very low wind conditions therefore making a balloon a better option. By using one of the most simple methods of aerial photography and data collection, we were able to gain an appreciation for modern unmanned aerial vehicles which can be programmed to survey a given area automatically.

Study Area

This activity was carried out at the Eau Claire Soccer Park in Eau Claire, Wisconsin. The facility, consisting of mainly soccer fields, indoor sports complex and a parking lot, also contained a concessions building, small playground and a basketball court. A picture of the study area is provided below courtesy of google earth.  The weather changed from overcast with calm winds at the beginning of data collection to overcast with a light rain and stronger winds to the West near the end of data collection. The goal was to survey the whole area between the sports complex on the West end to the tree line on the East end, however, the weather conditions in the end did not allow for the completion of this goal.

Figure 1: The area studied is outlined in red with the red arrow depicting the direction of North 

Figure 2: Inflating the balloon with helium

Methods

The first step in collecting the data for this activity was the assembly of the necessary apparatus. The first step in this process involved inflating a large balloon (Figure 2) with helium which would lift the camera rigging and GPS into the air. The balloon was filled until to a point where it had enough buoyancy to lift the camera rigging, GPS and tether into the air with ease. Once it was determined that this criteria was met, the balloon was removed from the helium source and the neck of the balloon was folded over a metal ring while one person pinched right above the ring so as to not release any helium from the balloon in the process. This metal ring would be used to tie the tether to.

Figure 3: Final set up of balloon apparatus

 Two zip-ties were then tightly fixed above the ring, holding the folded over balloon neck  tight so that helium would not escape and the ends of the ties were clipped off to avoid any danger of them puncturing the balloon, while giving the balloon a nice, clean look. Finally a long tether from a spool was attached to the ring to be used to adjust and maintain the height of the apparatus.Figure 3 depicts the final product of the balloon set up.





The next apparatus in need of assembly was the picavet rig. A picavet rig is a system consisting of an X-shaped platform for mounting camera/GPS equipment attached to series of cords used to stabilize the platform and is used in aerial photography to mount a camera to objects such as a kite or in our case a balloon (see figure 4 for usual picavet design). The picavet rig we used (Depicted in figure 5) contained a metal platform with two camera mounts and a strip of Velcro in the middle to attach the yellow GPS.  The hook of the rig were attached to the tether of the balloon several feet below the ring and was leveled out so that the cameras were facing straight down at an angle of 90 degrees, also known as nadir.

Figure 4: Usual picavet set up
http://www.kapshop.com/How_To/index.php?page=picavet-lacing

Figure 5: Picavet rigging used in our activity

The two cameras on the picavet rig were cannon point and shoot digital cameras, one regular camera and one that was modified to be partially infrared, and both had CHDK or Cannon Hacker Development Kit firmware installed on it. What CHDK does is modify the camera on top of its original functionality to allow the user to take time lapse photos or photos at a set rate. This is especially useful for aerial photography when you cant be in physical contact with the camera for an extended period of time. For our purposes while walking with a balloon, we decided to take pictures every 6 seconds. With a faster aircraft you could modify that time to take pictures faster since you will be covering more area faster, or you can set up your camera in a way to take pictures in increments dependent on a certain distance you travel. Both cameras were set to this setting and once they began taking pictures, the balloon was raised 100' into the air. A DJI Phantom drone was used to capture the video below of the rig in action.

When the balloon was at it's maximum height, we began walking the field starting at the East edge and moving North to South. At the end of the field we paced out 30' to the West and began walking the opposite direction until the end of the field (beginning of a sidewalk). Again we paced 30 feet to the West and proceeded back the opposite way we came. The purpose of going only 30' to the West instead of pacing out the entire focal range of the camera was to allow for enough overlap in the pictures so that they could be stitched together and analyzed  This process continued for about an hour when we reached the concession building and it began to rain. At the end we walked back to our base in the parking lot and disassembled the rig.

Discussion

The vertical or nadir view images that were captured are especially helpful for measuring and mapping the area of study. At this point in the curriculum we have not yet discussed how these images can be processed and interpreted, and they have yet to be stitched together, but below in figures 6 and 7 you can see the different types of photographs that were taken in both visible light and near infrared both of which have their own advantages.

Figure 6: Visible light imagery

Figure 7: Near infrared imagery

Visible light imagery is useful for plotting a map using colors people are more familiar with and gives the imagery a google earth "satellite view" feel. The near infrared imagery, while still able to be stitched together to form a recognizable map has purpose way beyond being a unique looking picture. Infrared imaging can be used to discover many things such as monitoring plant growth/health, the heat radiating off the rooftops of homes, and even detecting oil spills. With this near infrared photograph you can use the colors of the plant life as a baseline and use subsequent imagery on a later date to visualize a change in infrared light to monitor changes we wont be able to see with only visible light.

The next data that was collected was from the GPS module that was also attached to the picavet. This data was plotted into a Google Earth file to show the path that the balloon was taking while we were walking the field.

Figure 9: Example of the 3D nature of the Google Earth Plot

Figure 8: Google Earth map depicting the route of data collection

From these pictures we can see that the path the balloon was not a straight vertical line, which could cause problems with image stitching on top of the fact that the camera was not fixed, but allowed to freely spin during data collection. Also note that the spacing started to expand halfway through. This was the point in data collection where we began to worry about the rain and sped things up by spacing our paths out. Spacing the lines out could also provide it's own challenges with image stitching. In figure 9, you can see that the plot on Google Earth is in fact three dimensional. This depicts the altitude of the picavet rig at any given time during collection, which is an interesting feature to note.

Conclusion

Image gathering with a simple aerial apparatus like the balloon used in this activity has both its advantages and disadvantages. On the downside, the camera equipment isn't fixed and therefore freely swings and spins in the air making image stitching (I'd assume) more difficult than using a straight flying airplane with a fixed camera. Another down side is the balloon apparatus would probably only be practical in an open field setting as the stings could be easily tangled in an area with trees or larger structures. On the plus side, however, this apparatus is simple, way cheaper than a drone that could carry the equivalent equipment and an easy introductory method for collecting aerial photographs. That being said, now that I have an appreciation for the process of collecting aerial photographs, I look forward to learning more advanced methods and how to process the collected data. Stay tuned!