Contact us at 319-277-0436 or info@aerialservicesinc.com

Just How Accurate is Your Drone?

Just How Accurate is Your Drone?

Article Written by Mike Tully

Unmanned Aircraft Systems (UAS) continue to influence the profession of remote sensing and mapping 
like few things ever have. Advances in computer technology, global positioning, and miniaturization have conspired to remove considerable barriers to entry. Many new practitioners are buying drones and providing these services and data for the first time. Much (not all) of the science and art of photogrammetry is now coded on a chip. These advancements enable new practitioners to provide a greater array of services to new and existing markets than ever before and fosters the misperception that “anyone can do it”.

New practitioners of drone-based remote sensing and mapping need to understand the fundamentals of remote sensing, mapping, photogrammetry. Typical deliverables like orthophotography, digital elevation models (DEM), contours, cross-sections, and 3D models depend on this understanding. Nescience of these fundamentals is certain to cause considerable pain, financial loss and compromises to public safety. This article introduces the fundamentals of positional accuracy to help new practitioners provide these services consistent with professional accuracy standards.

I have talked with several practitioners that did not know what “ground control” was or how to use it to establish Truth #1positional accuracy. This lack of familiarity is not uncommon among novices. They may not know that positional accuracy requirements are needed, or that they are often assumed by the client. They may not know how to discuss positional accuracy with their clients, nor how to measure the positional accuracy of their deliverables.

Truth 1: Positional accuracy doesn’t just happen.

Professionals know that an accurate ortho (or DEM or 3D model) can look identical to an inaccurate one. Both are “pretty” pictures with lots of great detail, but one has more intrinsic value for a greater number of uses than the other because it is more accurate.

Truth 2: Positional accuracy is a product of the entire drone “system” (aircraft, sensors, operation, and processing software) not any single component.

Truth #2 for Mike's Drone Article Jan 10It matters very little what the drone vendor says about the positional accuracy of its products. A combination of factors (and seldom a single factor) affects the positional accuracy of an orthophoto, DEM, or other derivative of remotely sensed data. Poor operation of the best drone can vitiate the positional accuracy of a deliverable. If a drone manufacturer claims their camera is accurate to two pixels for any given ground sample distance (GSD), the resultant positional accuracy for the orthophoto is dependent on each of the following factors. [The list below is not a comprehensive list of error sources but includes the major contributors of error.]

  1. the cameras inherent potential accuracy
  2. the stability of the flight
  3. the quality of the GPS data
  4. the quality of the inertial system (if the drone even uses one),
  5. the quality of the DEM used to make the orthophoto, and
  6. the type and quality of processing of the raw imagery into an orthophoto (this factor alone has several important sources of error from a “raw” to “finished” product)
  7. the number and quality of ground control points

Truth #3 for Mikes Drone Article JanEach factor contributes some error to the ultimate positional accuracy of the final product. The sum of all errors
determines the measurable positional accuracy.

Truth #3: Positional accuracy standards exist and are important.

Understanding accuracy and accuracy standards sets your operations apart from others’. The American Society of Photogrammetry and Remote Sensing (ASPRS) is the major “standards body” for this profession. Their Standards for Geospatial Data reflect the realities of new sensors and digital data. They are “scale- and technology-agnostic”. That is, the standards apply to data produced at any scale using any kind of sensor today or tomorrow. They can be used to measure and report the positional accuracy of geospatial deliverables like orthophotography, DEMs, digital surface models, 3D models, contours, topographic mapping, etc.

Deliverables with good, consistent positional accuracy can be an important differentiator for your drone-based remote sensing business. Unfortunately, a main cost driver of Truth #4 for Mike's Drone Article Jan 16geospatial deliverables is positional accuracy. More accurate data will generally be more expensive than less accurate data. Profitability is highest when the required accuracy is not “over-engineered” and drives up costs.

Truth #4: Best possible positional accuracy today has error of 1 to 1.5 pixels (RMSE).

What level of positional accuracy is achievable using today’s drone systems? Assuming “best practices” with a drone using a metric camera (most drones do NOT have a metric camera), high quality ground control, and solid production procedures (all difficult to achieve consistently) the best possible accuracy for orthos would have a root mean square error (RMSE) = 1 to 1.5 Pixels (GSD). Are these levels of accuracy achievable flying a drone with a non-metric camera and without any ground control?  Not a chance … not today!

Because increasing accuracy comes at a premium it is imperative that the practitioner understand what accuracy is achievable from their drone “system”, what the client expects, and what is needed (this is often at odds with client expectations) to meet the deliverable’s intended use. Because quality remote sensing products and services are difficult to deliver and need considerable expertise that is not yet programmed into the “easy button”, many drone fliers are choosing to collect data and have established firms like Aerial Services produce positionally accurate, irrefragable geospatial deliverables.

Please contact us if we can help. Call 319-277-0436 or click here.

Mike Tully has been the President & CEO of Aerial Services, Inc. (Cedar Falls, IA) since 2002. He is a certified photogrammetrist, and has a B.S. in Forestry from Northern Arizona University, a M.S. in Forestry Entomology from the University of Maine, and a Masters in Business Administration from the University of Northern Iowa. He is a certified GIS-Professional, techno-geek, and the head of "Getting Right Things Done Well" at Aerial Services. Mike is an innovator and thinker.

4 Comments

  1. Gerrie DeGross 2 years ago

    Great write-up Mike!
    As a practicing photogrammetrist with 48 years of experience, it gives me great hope for the future when I see an honest artical published that brings to light the importance of accuracy into the equation for UAV data acquisition.

    NOTE: The following may be too lengthy for your review. However, it supports your artical with actual test results on data provided by uninformed or ill-qualified UAV providers.

    Today I see many of my surveying and engineering clients purchasing UAV’s to support their businesses. This, in and of itself is understandable. However I am not seeing the hiring of trained or experienced photogrammetrist to manage these UAV programs, thus assuring the reliability and accuracy of their services. As with all other disciplines in the surveying and engineering community it is my belief that qualified individuals with experienced photogrammetric backgrounds are a must in the acquisition of any UAV data. Without this, my fear is that the “easy button” as you refer to it, is and will be the tool of choice within these organizations. — Big mistake!–

    I have recently had the occasion to work with some low altitude DTM and ortho UAV data performed over a gravel pit site of a long time client. To the untrained eye of the client, as you mentioned, the finished products of this UAV data appear to be very good. The image is sharp, the data is dense and the contours flow in the anticipated manner, although very jagged and unreadable. The client was of course pleased with the lower costs, impressed with the density of the data and the image quality within his open pit site. However he was dismayed that the data was not and would not be provided in his desired AutoCAD format to which he was accustom, nor did het like the erratic shapes and bubbles portrayed by the contours or the blurred image of the ortho in the vegetated areas.

    Result:
    Our firm was hired to convert this data into AutoCAD format, analyse it for accuracy, add additional ortho imagery around the site and report on our findings.
    1) Converting the raw DTM data to AutoCAD was an easy input-output translation once the observation coordinates were converted to ground coordinates.
    2) Accuracy checks were made by evaluating: a) errors in long held premarked ground control points and b) residual errors as compared to existing photogrammetric mapping.

    Findings:
    In OPEN areas
    Horizontal accuracy: a standard error ~ 2 feet, maximum error in open ground 5′
    Vertical accuracy: standard error ~2′, maximum error in open ground 5′
    Image quality: Very good in open ground

    in VEGETATED areas:
    In vegetated areas of brush and trees this UAV data was basically worthless. The dense DTM data was so unreliable a detailed analysis was not even performed.
    The resulting ortho was a total smear with no recognition.

    Conclusion of the analyses:
    The provider of the UAV data obviously did not take appropriate steps to assure NMAS were observed in providing the data to the client, or the provider was not aware of the consequences of using the “easy button”. The provided DTM data was so dense and unmanaged that it did not in any way represent the surface model around raised objects or within vegetated areas.

    Warning to Client if accuracy and readability are important:
    1) Data density is not directly related to archived accuracy
    2) Some data thinning and last return surface analysis is required to eliminate DTM points obtained on above ground features and in vegetated areas.
    3) Unreliable and dense data acquisition in vegetated areas results in useless and undesirable blurred ortho image. Again , last return analysis of the DTM data is needed in such areas to obtain a reasonably accurate and sharp ortho image.
    4) Use data from this provider with caution if accuracy and presentation are desired.

    Warning to provider or would be provider:
    Educate yourself on the principals of photogrammetry and the science of resection and intersection
    Understand your products accuracies and inform you client of the accuracy your are able to provide
    Educate yourself on the opportunities available to you to enhance your services with increased accuracies, quality, and image displacement.

    HOPES FOR THE FUTURE:
    More opportunities for trained photogrammetrists.

    • Author
      Mike Tully 2 years ago

      Gerrie,
      Thank you for the kind words. Your experience is increasingly shared with other remote sensing professionals. Although the “easy button” included with more and more geo-software used to create point clouds and orthos is incredibly sophisticated and is certainly a welcome step toward “faster, better photogrammetry”, at this time the inexperienced, uninformed user should be cautious. Even after the new user is “trained” on how to use the software, they may not understand photogrammetry. Because “bad” data can easily look “good” to the untrained eye, poor quality remote sensing data from drones is easily output. It is incumbent on all professionals to educate new users and clients in the many new, and existing, markets about these issues so everyone comes out ahead.

  2. Antoine 2 years ago

    This is so true!

    As a provider of a structure from motion solution, we strive to make it automated, intuitive to use AND accurate. The danger, as you point out, is that by democratizing photogrammetry and putting it into the hands of anyone, some end users do not understand accuracy . Photogrammetrists and surveyors always do of course. For this reason we have developed a knowledge base (support.pix4d.com), do constant webinars, how-to videos and articles, and provide technical support – those work very well but require a lot of work.

    An article like this one is good as it educates new drone users on accuracy.

    • Author
      Mike Tully 2 years ago

      Antoine,
      My only critique of what you said is that you are too generous to photogrammetrists and surveyors! 🙂 We have used Pix4D and have found it to be an excellent product. When we have run into problems, technical support has always been very responsive. Novices to remote sensing and mapping are encouraged to take the time to really understand the photogrammetry behind the imagery and point clouds so products meet or exceed customer expectations.

Leave a reply

Your email address will not be published. Required fields are marked *

*