Creating Models Of Custom Image Classification Workflows Using Machine Learning Techniques
1Harris Geospatial Solutions Inc., U.S.A.; 2Harris Geospatial Solutions GmbH, Deutschland
Image classification is a well-known technique used in remote sensing to map land cover types and to monitor changes at the Earth’s surface over time. Most image processing software applications include multiple classifiers, along with the ability to display results and to evaluate the accuracy of the classification. In this study, a visual modelling tool was used to build multiple classification workflows using machine learning techniques such as Softmax Regression (SR), Support Vector Machine (SVM), and Random Forest (RF). The modelling tool simplifies the process of building custom classification workflows without requiring any knowledge of a specific application programming interface (API).
The study area chosen for this research is in eastern Texas, USA, along the Brazos River. The site was chosen because it contains a mix of land cover classes that are predominantly vegetative, including wetlands, forest, agriculture, and rangeland.
A Sentinel-2A Level-1C image of the study area was used for image classification. Additional processing steps were taken to prepare the image for analysis: (1) Layer stacking the visible and near-infrared bands, while resampling the 20-meter bands to 10 meters, (2) defining a spatial subset around a specific area of interest, and (3) masking all non-vegetation features such as roads, water, and buildings. A first-order entropy texture image was also created from the red band of the Sentinel-2 image.
Training data consisted of samples of Sentinel-2 image pixels that represent the following feature types: Nonforested wetland, Rangeland, Forest, Grass/Pasture, Cropland. Distinguishing between these land-use types using remote sensing imagery can be challenging, especially since many areas contain a mix of land-use types. In seldom cases the categories are defined by abrupt boundaries, except for possibly cropland.
Several models were built to execute and compare the results of multiple classifiers. Supervised classification based on machine learning typically requires extra steps such as normalizing and randomizing the input data, training the classifier, and minimizing loss. The modelling tool incorporates these elements into the classification process. Users only need to define the trainer (such as gradient descent or iterative) and the classifier (such as SR, SVM, and RF) that will be used.
Each model was run using 10 spectral bands as input to the classification, and again with an entropy texture layer as additional input, to determine if texture improved classification accuracy.
This study demonstrates that models facilitate the comparison of different classifiers while allowing an analyst to experiment with different input features. Building a model with a visual programming tool has the benefit that inputs, outputs, data management operations, and processing tasks can be linked with a drag-and-drop user interface instead of learning API code. Models can be packaged and deployed to desktop and cloud-computing environments for reuse and further customization. Model files can also be incorporated into larger image-processing models.
FORCE – Analysis Ready Data and beyond
Humboldt-Universität zu Berlin, Deutschland
We are currently experiencing an exciting new era of Earth Observation, wherein multiple, freely available remote sensing systems provide us data at unprecedented spatial, temporal and spectral resolutions. This regular data influx might enable us to achieve sustainable development goals by closely monitoring environmental status and change at relevant scales. However, this flood of data can easily be overwhelming, both in terms of volume and usage complexity. While the first point is merely a technical burden that can be leveled out with enough processing power and investments in storage, EO data still need to be processed to a considerable degree before being adequate for most analyses.
To alleviate this, the Framework for Operational Radiometric Correction for Environmental monitoring (FORCE) is being developed to be an ‘all-in-one’ solution for the mass-processing of medium-resolution satellite image archives to enable large area and time series applications. FORCE is capable of processing Landsat 4–8 and Sentinel-2 Level 1 data to Level 2¬–4 products, which are essentially different degrees of Analysis Ready Data (ARD). Such data can be analyzed with a minimum of additional user effort, which will empower much broader user groups in using mass data from Landsat and Sentinel-2.
Currently, the term ARD is mostly used to describe data that are radiometrically and geometrically consistent, and that include cloud and other poor quality observation flags that allow filtering data prior to analysis. This definition is well aligned with FORCE Level 2 ARD, which are essentially radiometrically corrected Bottom-of-Atmosphere reflectance products. FORCE Level 2 data are accompanied by pixel-based quality information and are stored in a gridded data cube using a single projection across large areas.
Whilst these data can indeed be directly analyzed, usage is complex nonetheless. Data volume is still similar as before, and Level 2 ARD are spatially incomplete (due to observation characteristics, and quality screening). Therefore, FORCE provides a Level 3 Processing System that generates temporal aggregations of ARD to provide seamless, gap free, and highly Analysis Ready Data (hARD) over very large areas. This includes Best Available Pixel (BAP) composites as well as Spectral Temporal Metrics (STM, e.g. average reflectance within a defined time period). Unlike ARD, hARD are considered the optimal input for many machine learning applications, e.g. for large area land cover /change classification purposes.
Level 4 data are defined as “model output or results from analyses of lower level data, e.g., variables derived from multiple measurements” (Asrar and Greenstone 1995). To this end, FORCE provides the Time Series Analysis (TSA) module that is capable of generating and analyzing quality-controlled time series from spectral bands, indices or unmixed fractions using a number of aggregation and interpolation techniques. Annual Land Surface Phenology (LSP) metrics can be derived, and change and trend analyses can be performed on any of the generated time series. Many outputs of FORCE TSA are considered as highly Analysis Ready Data plus (hARD+), meaning that generated products can be opened in a GIS and be directly analyzed to fuel your research questions without any further processing.
Asrar, G., and R. Greenstone, eds (1995). MTPE EOS Reference Handbook. Greenbelt, MD, USA. NASA/Goddard Space Flight Center.
Using Time Series Information For Mapping Human Settlements With Sentinel-2
Humboldt-Universität zu Berlin, Deutschland
Mapping human settlements with remote sensing is particularly challenging. The heterogeneity of urban areas and the complex interactions with their surrounding environment require both an understanding of urban processes and a solid methodology to accurately quantify urban land cover. Sentinel-2 offers free and globally available optical remote sensing data and provides improved spatial, temporal and spectral resolution compared to similar previous and existing sensors. Its use is, thus, considered to have a huge potential to contribute to urban monitoring.
Even if artificial materials are of most interest in urban surface mapping, urban spaces are also largely characterized by vegetation. Vegetation is present in urban green areas and street green as well as surrounding natural green areas, urban forests or even agriculture in close distance to cities. The quality of urban land cover mapping is, thus, dependent on temporal image availability and selection, because vegetation phenology has an impact on land cover patterns throughout the year.
This study shows that information from Sentinel-2 image time series is potentially able to enhance the quality of mapping urban land surfaces in the context of vegetation/imperviousness/soil detection. Our land cover mapping methodology is based on a machine learning support vector regression approach trained with synthetic spectral mixtures generated from image spectral libraries containing pure surface cover types. We compared model performance using four seasonal Sentinel-2 images (Spring, Summer, Fall, Winter) as well as derived time series metrics (e.g. spectral percentiles or vegetation indices) for the area of Berlin, Germany, as input data. We systematically evaluated all model results in ten selected regions of interest with different neighborhood characteristics within the city of Berlin and the urban fringe. Reference data for validation was composed of publicly available cadastral information and manually digitized urban features.
We find that spring and summer imagery as model input perform better than fall and winter imagery. In the latter case, there is a particular conflict between urban vegetated areas and soils as well as impervious surfaces. Spring and summer imagery work well within the urban core, but performance decreases at the urban fringe, where uncultivated agricultural fields show high fractions of imperviousness. This issue is resolved with time series information. When all cloud-free data available for each pixel within one year is accounted for, model performance in comparison with spring and summer models is stable in the urban center and further increases in the outskirt areas, where surface distinction is clearer with time series metrics.
The study shows that using Sentinel-2 imagery, different time series metrics computed from a one-year-period can resolve the need to find a best observation for urban mapping without requiring complex image compositing. Time series information might be particularly useful to regionally generalize regression models and apply urban mapping methods to different world regions and larger areas.
Snapshot Hyperspectral Imaging for Field Data Acquisition in Agriculture
1Szent István University, Hungary; 2Leipzig University, Germany; 3NARIC Institute of Agricultural Engineering, Hungary; 4Brightic Research GmbH, Hungary
With two case studies the research benefits and potential of snapshot hyperspectral imaging will be presented here. Traditionally, point spectrometers were the first choice in field spectroscopy, occasionally complemented by hyperspectral field scanners and recently by snapshot imaging spectrometers. Scanners show some limitations when moving targets or rapidly changing processes have to be captured. The snapshot advantage reveals these limitations and opens novel application fields. We used and tested a snapshot spectral camera with more than 100 spectral channels in two novel agricultural research projects.
A native snapshot imaging spectrometer captures all spectra and the entire image at the same time without any time delay. It enables this imaging system to capture motion pictures and producing hyperspectral videos. We used an UHD 185 (Cubert GmbH, Ulm, Germany) snapshot video spectrometer (400-1000 nm, 100< bands) to visualize 'floating dust' known as particulate matter (PM). The EU introduced limits for particulate matter (PM10) in 2010. PM10 is particulate matter with 10 micrometres or less in diameter, inhalable into the lungs and thus relevant as risk for human health. Agricultural tilling operations are important PM10 sources affecting air quality and soil fertility (loss of bound organic material and nutrients); additionally, soil particles suspended in the air have marked effects as crystallisation nuclei being relevant for the formation of precipitation. We measured three different agriculture activities using a standard micrometeorological dust measuring station. Results have been compared to real-time spectral videos to describe rapid dust concentration changes in space and time behind the tractor. Using a hyperspectral video spectrometer tilling dust could better be visualized and classified compared to non-imaging methods, which seems to be a promising tool for aerosol mapping and field documentations. We report about our results, challenges and experiences made with video spectroscopy.
In our second case study the snapshot camera was applied to spectrally document, map and characterize a raspberry plantation under differently coloured shade nets to analyse usability, flexibility and to record spectro-phenological parameters. Leading raspberry producers are located in Eastern Europe and contribute at least 80% of the world's raspberry production (FAOSTAT, 2016). Due to agricultural climate change scenarios raspberry plantations are at risk because evapotranspiration will be challenged by solar radiation and temperature changes.
Advanced spatial sampling solutions are needed to characterize radiation/reflectance properties of the plantation and to evaluate the success of preventive shading techniques. In agricultural high mobility, flexibility, weight and speed are of high importance. Snapshot hyperspectral imaging works as a non-invasive and non-destructive sampling method with extreme short data acquisition time. As a hardware test camera usability, data delivery, spectral data quality and spectral documentation potential were evaluated.
We concluded that spectral field data acquisition and length of data evaluation could be significantly reduced by snapshot spectral imaging. Data prototyping and pre-processing could be speeded up from days to minutes. Snapshot imaging spectroscopy is a complementary data acquisition tool for agricultural professionals to gain more accurate physical, phenological and physiological information on radiation induced environmental changes. This information helps manage light supply, optimize shading and crop managing strategies in order to maintain quality production, health safety and market attendance.