A Comparative study of the traditional accuracy assessment and disagreement measures of the classification of remote sensing imagery

Document Type : علمی - پژوهشی

Authors

Department of Geomatics Engineering, Faculty of Civil Engineering and Transportation, University of Isfahan

Abstract

Evaluation of the image classification results is very important in the remote sensing projects. So far, many indices have been presented to assess the accuracy of image classification, though Kappa coefficient and Overall accuracy are the most famous ones. Some researchers have criticized these two parameters, and have presented new parameters for evaluation of the classification results. In this paper, the relation between two new accuracy assessment parameters (presented by Pontius & Millones) and traditional accuracy assessment parameters (Overall accuracy and kappa coefficient) is studied. These two new parameters are called “Quantity disagreement” and “Allocation disagreement” which report disagreement between ground truth and classification data. In order to apply the comparative study on the traditional and new disagreement measures, supervised maximum likelihood classification was applied on 57 satellite images with different spatial resolutions. Then, Kappa and Overall accuracy as traditional accuracy parameters and Quantity disagreement and Allocation disagreement as new measures were computed for each classified image and then the correlation coefficients of the both measures were calculated. The results show a high correlation between new parameters and traditional ones in negative direction irrespective the spatial resolution. In this way, the disagreement do not provide new information about the classification results to the user, and only if there is any request for classification error, the new disagreement parameters can be used along with the traditional ones.

Keywords

References

  1. Congalton, R.G. & Green, K., 2008, Assessing the accuracy of remotely sensed data: principles and practices: CRC press.
  2. El-Aziz, M. A. E.-A., 2004, Evaluation of soft classifiers for remote sensing data.
  3. Emami, H., 2005, Introducing correctness coefficient as an accuracy measure for sub pixel classification results.
  4. Foody, G.M., 2002, Status of land cover classification accuracy assessment, Remote sensing of environment, 80(1), 185-201.
  5. Foody, G. M. & Arora, M.K., 1996, Incorporating mixed pixels in the training, allocation and testing stages of supervised classifications, Pattern Recognition Letters, 17(13), 1389-1398.
  6. Gu, J., Sun, G., Pan, Y. & Fan, D., 2012, Accuracy assessment based on the distribution of the classified errors on classified map: Simulation analysis. Paper presented at the Agro-Geoinformatics (Agro-Geoinformatics), 2012 First International Conference on.
  7. Hashemian, M., Abkar, A. & Fatemi, S., 2004, Study of sampling methods for accuracy assessment of classified remotely sensed data, Paper presented at the International congress for photogrammetry and remote sensing.
  8. Li, W. & Guo, Q., 2014, A new accuracy assessment method for one-class remote sensing classification. IEEE Transactions on Geoscience and Remote Sensin.
  9. Marpu, P. R., Wijaya, A. & Gloaguen, R., 2008, Soft classification and assessment of kalman filter neural network for complex landcover of tropical rainforests, Paper presented at the Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008. IEEE International.
  10. MASELLl, F., Rodolfi, A. & Conese, C., 1996, Fuzzy classification of spatially degraded Thematic Mapper data for the estimation of sub-pixel components, International Journal of Remote Sensing, 17(3), 537-551.
  11. Mathers, P., 1999, Computer Processing of Remotely-Sensed Images: John Wiley & Sons.
  12. Persello, C. & Bruzzone, L., 2010, A novel protocol for accuracy assessment in classification of very high resolution images, IEEE Transactions on Geoscience and Remote Sensing.
  13. Pickard, B., Gray, J., & Meentemeyer, R. (2017). Comparing Quantity, Allocation and Configuration Accuracy of Multiple Land Change Models. Land, 6(3), 52.
  14. Pontius Jr, R. G. & Millones, M., 2011, Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment, International Journal of Remote Sensing, 32(15), 4407-4429.
  15. Pontius, R. G., 2000, Quantification error versus location error in comparison of categorical maps, Photogrammetric Engineering and remote sensing, 66(8), 1011-1016.
  16. Richards, J.A., 2013, Remote sensing digital image analysis: Springer.
  17. Story, M. & Congalton, R.G., 1986, Accuracy assessment: a user’s perspective, Photogrammetric Engineering and remote sensing, 52(3), 397-399.
  18. Strahler, A.H., Boschetti, L., Foody, G.M., Friedl, M.A., Hansen, M.C., Herold, M., . . . Woodcock, C.E., 2006, Global land cover validation: Recommendations for evaluation and accuracy assessment of global land cover maps, European Communities, Luxembourg, 51(4).
  19. Sui, C., Tian, Y. & Xu, Y., 2014, An Unsupervised Band Selection Method Based on Overall Accuracy Prediction, Paper presented at the Pattern Recognition (ICPR), 2014 22nd International Conference on.
  20. Warrens, M.J., 2015, Relative quantity and allocation disagreement measures for category-level accuracy assessment, International Journal of Remote Sensing, 36(23), 5959-5969.
  21. Wilkinson, G.G. 2005, Results and implications of a study of fifteen years of satellite image classification experiments, IEEE Transactions on Geoscience and Remote Sensing, 43(3), 433-440.
  22. Yan, G., Mas, J.F., Maathuis, B., Xiangmin, Z. & Van Dijk, P., 2006, Comparison of pixel‐
Iranian Journal of Remote Sensing & GIS
Volume 10, Issue 4 - Serial Number 4
February 2019
Pages 55-68

Files

Share

How to cite

Statistics