Principles of Remote Sensing and Mapping Using Drones

Since the FAA Part 107 rules were released, the future of UAS commercial data collection is looking up. Although the size of the UAS market under Part 107 is a mere fraction of what it will become once beyond visual line of sight flying (BVLOS) becomes a reality. Today there are some mapping applications well suited to drones. New applications tailored for Drones are being developed too.

It goes without question that drones have dramatically lowered the barriers to entry into remote sensing and mapping. Increasing numbers of new remote sensing and mapping practitioners flying drones are unfamiliar with photogrammetry and other principles of remote sensing and mapping. As with any new technology while in the “wild west” stage of development, unfounded, erroneous assumptions are made by practitioners about the positional accuracy of their mapping products.

This article is a discussion about the principles of remote sensing and mapping and how they apply to drones. Specifically, it will answer the question “What is positional accuracy and how do I know when I have it with my UAS map products?”

Principal #1 – Positional accuracy is the final product of detailed planning and diligent work.

Accuracy does not just happen. It is not purchased with the drone. It’s hard work and is based on principles that don’t change. The nature of things does not change simply because a new guy is flying a new contraption up in the sky! It is manufactured. It is a value-add of the drone services provider.

It requires skill and careful, informed planning.  It requires knowledge about the concepts in this article.

Principal #2 – All measurements contain error.

Consider a simple case. The most accurate ruler cannot measure distance without some error.  This ruler is only good to a 1/10th of a unit, maybe a 20th if you interpolate between two marks.

A more accurate ruler does exists that can be used to quantify the error of the measurements made with the less accurate ruler. But even if we understand this ruler’s inherent limit to measurement error, it could still be used ineptly to provide unmerited assurances of accuracy. Accuracy may be overstated and not supported by the limits of ruler.

For example, it is not uncommon to hear desktop GIS users explain that the application displays location values to the 10,000th of a unit and therefore the measurement is that good! But it’s not. The accuracy of this position, or distance, or size in your geospatial application is determined by the quality of measurements NOT the resolution of your software. The “ruler” does not convey “accuracy” to your data.

 So just like with the ruler, if I claim that my data is good to the 10,000th because my GIS says so, then I am overstating the nature of my data and misusing this “ruler” to characterize my data without consideration of its inherent limitations. The adage certainly applies with drones and remote sensing: garbage in -> garbage out.

As this profession rides through the “wild west” stage of drone use, many practitioners don’t even know they are producing garbage and trust that because the “ruler” says “good to 2 cm” that their mapping products are inherently that accurate.

Principal #3 – The entire drone system determines the total positional error.

It is important to appreciate that your drone “system” is making many measurements.  The drone system is comprised of all the drone sensors and applications.

These include sensors like the camera, gps, and inertial unit. But other components of the system include the flight management system and software used to produce deliverables from the collected data. As stated in Principle #2: Each observation by each component of the system is “erroneous”.

All of this error accumulates and can easily become excessive if not managed thoughtfully. Each component and how the “flying instrument” is operated affects the total error of the system. The following list is a very brief list of things that can contribute to the quality (error) of resultant deliverables:

• Aircraft performance: pilot skills, weather, flight management system, stability
• Pilot skills: the best, most accurate system in the wrong hands will produce junk
• Weather: wind, visibility, ground conditions all effect the accuracy of the “system”
• Ground control point quality
• Processing software and data processing methods

Rolling Shutter problem

Typical DSLR cameras (using CCDs as opposed to CMOS) use a “global shutter”. That is, they capture an entire frame at the same instant. The shutter opens then closes in a millisecond or so. But other cameras (generally CMOS cameras that are used by many drone manufacturers) use a “rolling shutter” where the entire scene is captured in pieces across time. This type of shutter introduces distortions into imagery of fast moving objects. (In remote sensing and mapping applications, especially where we are flying at less than 400′ away from features, pretty much everything is always moving rapidly).

Drones are often equipped with cameras using rolling shutter. These cameras have many desirable characteristics for some applications. However, the rolling shutter effect on imagery when used for mapping can introduce large error. Unless this source of error is understood and properly managed the accuracy of deliverables may be significantly impacted. Doubly bad is that these effects in your products are not easily discernable. The orthos may look fine! Fortunately, Pix4d and other processing software do attempt to model out these distortions. But at the end of the day, the rolling shutter introduces more error into the drone system and needs to be managed adeptly.

Which type of camera are you using? Do you know? Many don’t.

Data Processing

The best software in the wrong hands will produce junk. All data processing requires considerable understanding of the separate processes, the nature of your data, and what products are being produced for what purpose. Something as simple as poor file management can mess everything up! Failure to grasp any of these things can negatively impact map accuracy.

Decisions about which software option to use are important and often differ depending on observations about the input data or the intended use of that data.  Some software makes tradeoffs between ease of use and the ability to manage the inherent error in the data and other processes. Did you know that? What trade-offs for ease of use and expedience is your software of choice using?

Drone manufacturer claims to accuracy are really limited to just the drone and maybe the packaged sensors. All the other “system” components affect the deliverable accuracy for your project as much or more than the inherent accuracy of the aircraft & sensors. These manufacturer claims really express a theoretical maximum accuracy. Your mileage will vary based on all the other components of the drone system and the skill at which that systems is operated.

Read the fine print!

Error Budget

All the components of the drone system introduce discrete sources of error. The typical project has some type of accuracy specification or “intended use” that expresses the client’s expectations of accuracy.

The entire drone system’s “error budget” needs to be quantified so you are confident the error budget is not exceeded, and so the accuracy specifications can be achieved … or, if so, help you determine that another more accurate drone (or sensor or technician) with a smaller cumulative error is needed.

Unfortunately, today many of our drone systems’ (hardware and software components) error budgets are unknown. They have not yet been objectively quantified. The only way to quantify these are to systematically test the positional accuracy of different drone systems.

Principle #4: Accuracy standards do exist and are important.

Remote sensing and mapping professionals know that our mapping products often must meet specific accuracy criteria per some accuracy standard.

That is, it’s up to the practitioner to quantify the error in the products created by our drone systems, and to ensure that error is less than some acceptable amount.

But many new drone practitioners don’t know that these standards exist, and don’t understand how to quantify the accuracy of their data.

There are several existing map accuracy standards that are commonly used. The new ASPRS standards were updated a few years ago for digital, scale-less data collected by all our sensors today. The United States Geological Survey (USGS) have their own standards. We still see the old National Map Accuracy Standards (NMAS) used in solicitations by uninformed clients. These are old, outdated, and inappropriate for modern data.

The NSSDA (National Standard for Spatial Data Accuracy) is often confused as an accuracy standard. It is not. Instead it is a prescribed method of “reporting” accuracy statistics that is referenced by many of the accuracy standards. Many clients will use accuracy specifications that dictate NSSDA reporting. NSSDA never sets an accuracy level. That’s what the USGS and ASPRS accuracy standards do. But those accuracy standards require that accuracy statistics be reported using NSSDA methods.

Principal #5 – Ground testing is always recommended for mapping (but may not be required).

Your new Sensefly drone says it can be meet a positional accuracy of +/- 5 cm. (making this up). But because of what we just talked about, how do you know your data from the “drone system” meets this level of accuracy?

Only a small part of the accuracy achieved in your mapping is determined by the aircraft and sensor. That being the case, how will you know the positional accuracy of your mapping products if you don’t test them?

Answer: “You can’t!!”

The only way to “test” accuracy of mapping data is to use ground control.

Unless you know from repeated testing of your drone system that it repeatably and reliably meets a certain positional accuracy, you have to test it. OR, you could tell your client that it “MIGHT meet the accuracy specification” but that you really don’t know. Manufacturer claims have LITTLE to do with the real world achievable accuracy of your mapping products.

You shouldn’t use the manufacturer’s tech sheet to quote your capabilities. It is aggravating to tell your boss that “I can only guarantee 10 cm accuracy” while he forwards you emails from service providers claiming 2.5 cm accuracy. Inform her about “achievable”, “repeatable” accuracy not “pie in the sky” accuracy.

The biggest point of all is: control, control, control. You can never have too much ground control. And typically, you can’t professionally afford to eliminate ground control because positional accuracy remains too uncertain.

Principal #6 – Checkpoint survey accuracy must be 3x more accurate than the required accuracy data

Careful positioning of each point within the project area and knowing the number of required points is critical. If the points are surveyed to 3x better accuracy but are not positioned well or are too sparse, the points may not sufficiently (statistically) “test” the positional accuracy of the mapping.

This figure is over-simplified. You will often have less rectangular shaped areas or other factors that may require more points. The important thing is to include ground control points around the perimeter of the project and 1 or 2 points in the middle. [Note: Those two points in the middle make an important difference in reducing positional error.]

Principal #7 –  All positional accuracy methodologies have limitations especially when the number of check points is small.

It is not always feasible to establish surveyed GCPs because of fragile terrain, unreachable areas, personal safety, or project budget constraints. However, statistical theory says that at least 20 well positioned GCPs are needed to estimate positional error in a population of observations.

Statistical theory says at least 20 check points are needed to estimate positional error in a population of data. Check points are used to sample the population of all error introduced by our drone system and estimate the cumulative error in our mapping products. But this estimate of “true error” is comprised of a small sample of points (20, for example). We can’t measure the positional error of every point and pixel because it’s cost-prohibitive. So we sample the error. And our estimate of “true error” is just that, an estimate. It contains “sampling” error. Therefore, statisticians tell us that at least 20 (well  positioned) points are needed to have any validity to our estimate of error.

We often use fewer GCPs, but when we do, we cannot make statistical claims about “error”. We can only infer things about the accuracy of our data. Some accuracy standards allow describing the positional accuracy of deliverables as either “tested” (a higher threshold) or “produced to meet” (a lower threshold). The “produced to meet” method is appropriate for mature or established technologies where established procedures for project design, quality control and the evaluation of relative and absolute accuracies compared to ground control have been shown to produce repeatable and reliable results. (ASPRS, 2015)

TakeAway

Project specifications are important and need to be carefully reviewed to determine the right tool for the job. Is a drone needed? Can I use a VTOL or should it be a fixed wing? What type of sensor is needed? What are the accuracy specifications? Will weather factors be a problem? Will the drone have sufficient stability in the expected (and unexpected) weather? Can it be accomplished under budget?

The takeaway for procurers of geospatial data is that anyone operating a UAS for mapping needs to be familiar with these foundational remote sensing and mapping principles and understand how to quantify errors that always accumulate in the mapping. If your UAS service provider is unaware of these concepts (like accuracy standards, NSSDA, ground control, error budget) than I’d look elsewhere for a service provider.

The takeaway for producers of geospatial data is to understand these remote sensing and mapping principles. Learn how to test or “produce to meet” the positional accuracy of your mapping products so you can reliably produce quality products.

Works Cited

ASPRS. (2015, March). ASPRS Positional Accuracy Standards. Photogrammetric Engineering & Remote Sensing for Digital Geospatial Data Ed. 1, Ver. 1.0 Nov 2014, pp. A1-A29.