Difference between revisions of "Data processing and output of Lidar"
(→Producing a DTM) |
Dronkers J (talk | contribs) |
||
| (15 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
| − | {{ | + | {{Review |
| + | |name= Andrea Taramelli | ||
| + | |AuthorID=20734 | ||
| + | }} | ||
| − | After [[use of Lidar for coastal habitat mapping|Lidar (Laser Induced Detection And Ranging | + | |
| + | After [[use of Lidar for coastal habitat mapping|Lidar (Laser Induced Detection And Ranging or LIght Detection And Ranging) has been used]], collected [[data]] needs to be converted to required outputs. This article describes the processes related to data post-processing, products and a '''Digital Terrain Model (DTM)'''. | ||
==Data post-processing and products== | ==Data post-processing and products== | ||
===Data quality check === | ===Data quality check === | ||
| − | Processing topographic Lidar data requires two major steps a) an internal quality check of the raw data and b) an external quality check. Quality checking mainly means making sure of three points: the data density (to ensure suitable DTM output) and the horizontal and vertical accuracy of data. | + | Processing topographic Lidar [[data]] requires two major steps a) an internal quality check of the raw data and b) an external quality check. Quality checking mainly means making sure of three points: the data density (to ensure suitable DTM output) and the horizontal and vertical accuracy of data. |
| − | Data density may not be complied with when the survey navigation is not properly carried out, resulting in gaps between adjacent swaths or over water patches (since water theoretically absorbs the | + | Data density may not be complied with when the survey navigation is not properly carried out, resulting in gaps between adjacent swaths or over water patches (since water theoretically absorbs the infra-red radiation). The operators usually provide a density map along with the data files. |
Quality control for topographic data involves getting rid of outliers which may be a result of some obstructions to the light path. It also means filtering objects of no direct interest for the user (typically vegetation, houses and any human objects) to arrive at a DTM representing true ground heights. | Quality control for topographic data involves getting rid of outliers which may be a result of some obstructions to the light path. It also means filtering objects of no direct interest for the user (typically vegetation, houses and any human objects) to arrive at a DTM representing true ground heights. | ||
While the ground signature can be retrieved fairly easily on topographic Lidar, the bottom signature is much more complex to detect on hydrographic Lidar. In practice, all groups of dots are individually reviewed by the interpreter who decides whether the bottom has been found or not. This also depends on the context (neighbouring dots) and examination of the waveforms. Light backscatter may be modulated by standing vegetation (e.g. sea grass or kelp) which can jeopardize bottom detection. | While the ground signature can be retrieved fairly easily on topographic Lidar, the bottom signature is much more complex to detect on hydrographic Lidar. In practice, all groups of dots are individually reviewed by the interpreter who decides whether the bottom has been found or not. This also depends on the context (neighbouring dots) and examination of the waveforms. Light backscatter may be modulated by standing vegetation (e.g. sea grass or kelp) which can jeopardize bottom detection. | ||
===Typical topographic Lidar products === | ===Typical topographic Lidar products === | ||
| + | [[image:lidar_tab_02.jpg|thumb|600px|left|Table 1: Typical output products from a topographic Lidar survey]] | ||
The first products the Lidar operator is asked to provide are triplets (xyz) expressed in an ellipsoid vertical reference and in any suitable horizontal coordinate system. Table 2 shows the various products. Data sets must have been post-processed into true height data without any outliers. In most cases this should be double checked using appropriate software capable of dealing with large data sets. While a gridded product may be produced by quick interpolation on a medium size mesh (typically 5 metres) for use as a data quick-look, the user may want a ready-for-use DTM grid file in the best possible resolution, e.g. a metric one. In this case, this must be specified at the outset of the work. More advanced prefer to produce their own interpolation (see Table 1). | The first products the Lidar operator is asked to provide are triplets (xyz) expressed in an ellipsoid vertical reference and in any suitable horizontal coordinate system. Table 2 shows the various products. Data sets must have been post-processed into true height data without any outliers. In most cases this should be double checked using appropriate software capable of dealing with large data sets. While a gridded product may be produced by quick interpolation on a medium size mesh (typically 5 metres) for use as a data quick-look, the user may want a ready-for-use DTM grid file in the best possible resolution, e.g. a metric one. In this case, this must be specified at the outset of the work. More advanced prefer to produce their own interpolation (see Table 1). | ||
| − | |||
===Typical hydrographic Lidar products=== | ===Typical hydrographic Lidar products=== | ||
| Line 30: | Line 34: | ||
which could provide an indication of bottom type (see "Lidar backscatter intensity" below). | which could provide an indication of bottom type (see "Lidar backscatter intensity" below). | ||
| − | [[image:lidar_tab_03.jpg|thumb|500px| | + | [[image:lidar_tab_03.jpg|thumb|500px|right|Table 2: Typical output products from a hydrographic Lidar survey (Admiralty coastal surveys)]] |
===Data accuracy=== | ===Data accuracy=== | ||
| Line 44: | Line 48: | ||
===Data validation=== | ===Data validation=== | ||
| − | Topographic Lidar accuracy is checked by way of high quality DGPS determination of reference surfaces. In practice, it is recommended to survey two of these references per aircraft sortie, basically one at the start and the other at the end. Typically, these surfaces should be smooth and rather flat, so that the horizontal inaccuracy has a limited influence on vertical accuracy. The best surfaces are playing grounds, with a large flat area surrounded by vertical objects (hedges, railings, posts). Validation is a two-step process. The horizontal positioning check should be carried out first: this is done by surveying a number (e.g. 30) of “vertical objects” in the field, namely their footprints on the ground. Once the horizontal accuracy has been shown to be within the specified limits (i.e. between 0.5 to 1 metre rms for topographic Lidar dots), the vertical check can be performed. A set of surveyed points no further apart than the Lidar dot spacing (i.e. about 3 to 5 metres) is selected. Lidar spots no more than one metre away are then chosen and paired with the ground points. Sufficient numbers of these pairs can then be processed statistically. The literature (Huising, 1998, Joinville 2002, Populus, 2003) shows than on a bare, smooth and reasonably sloped terrain, vertical accuracy better than 0.15 m rms is always obtained with topographic Lidar. These figures degrade with the terrain type, e.g. low-lying vegetation such as tidal marshes, and slopes as is the case in cliff type shorelines. | + | Topographic Lidar accuracy is checked by way of high quality DGPS determination of reference surfaces. In practice, it is recommended to survey two of these references per aircraft sortie, basically one at the start and the other at the end. Typically, these surfaces should be smooth and rather flat, so that the horizontal inaccuracy has a limited influence on vertical accuracy. The best surfaces are playing grounds, with a large flat area surrounded by vertical objects (hedges, railings, posts). Validation is a two-step process. The horizontal positioning check should be carried out first: this is done by surveying a number (e.g. 30) of “vertical objects” in the field, namely their footprints on the ground. Once the horizontal accuracy has been shown to be within the specified limits (i.e. between 0.5 to 1 metre rms for topographic Lidar dots), the vertical check can be performed. A set of surveyed points no further apart than the Lidar dot spacing (i.e. about 3 to 5 metres) is selected. Lidar spots no more than one metre away are then chosen and paired with the ground points. Sufficient numbers of these pairs can then be processed statistically. The literature (Huising and Pereira, 1998<ref>Huising E.J., Gomes Pereira L.M., 1998. Errors and accuracy estimates of laser data acquired by various laser scanning systems for topographic applications, ISPRS Journal of Photogrammetry and Remote Sensing 53, 245-261: pp. 17.</ref>, Joinville et al. 2002<ref name="join">de Joinville O., Ferrand, B., Roux M., 2002. Levé laser aéroporté : Etat de l'art, traitement des données et comparaison avec des systèmes imageurs, in : Bulletin SFPT N°166, pp 72-81.</ref>, Populus et al , 2003<ref>Populus J., Laurentin A., Rollet C., Vasquez M., Guillaumont B., Bonnot-Courtois C., 2003. Surveying coastal zone topography with airborne remote sensing for benthos mapping, eProceedings of Earsel's GIS "Remote Sensing of the Coastal Zone", Ghent, 2003 June 5-7, 105 117: pp. 13.</ref>) shows than on a bare, smooth and reasonably sloped terrain, vertical accuracy better than 0.15 m rms is always obtained with topographic Lidar. These figures degrade with the terrain type, e.g. low-lying vegetation such as tidal marshes, and slopes as is the case in cliff type shorelines. |
| − | With hydrographic Lidar, ground truth is much more complicated to achieve for obvious reasons: being underwater, the co-location of Lidar measurements and field checks is much more difficult. Places with bare sediment bottom and which can be safely accessed by boat must be chosen. Ideally, field data should be recorded with a shallow water multibeam sounder. It is crucial that all measurements be carried out with respect to a vertical system independent of the water levels (ellipsoid heights). Very few reports are available to date on validating bathymetric Lidar with acoustic data. | + | With hydrographic Lidar, ground truth is much more complicated to achieve for obvious reasons: being underwater, the co-location of Lidar measurements and field checks is much more difficult. Places with bare sediment bottom and which can be safely accessed by boat must be chosen. Ideally, field data should be recorded with a shallow water multibeam sounder. It is crucial that all measurements be carried out with respect to a vertical system independent of the water levels (ellipsoid heights). Very few reports are available to date on validating bathymetric Lidar with acoustic data. |
===Producing a DTM === | ===Producing a DTM === | ||
| − | Data interpolation is not dealt with here, since many suitable references can be consulted by readers. Some authors (Joinville, 2002, Daniels, 2000) give a good account of their procedure and guidance can be found there. However, operational documents fully describing these steps have not been produced yet. The following tips can be recommended: | + | [[image:lidar_fig_06.jpg|thumb|300px|right|Figure 1: Sketch of overlap]] |
| + | Data interpolation is not dealt with here, since many suitable references can be consulted by readers. Some authors (Joinville et al, 2002<ref name="join"/>, Daniels, 2000)<ref>Daniels, R.C, 2000. Datum Conversion Issues with Lidar Spot Elevation Data, Photogrammetric Engineering & Remote Sensing, Vol. 67, No. 6, pp. 735-740: pp. 17.</ref> give a good account of their procedure and guidance can be found there. However, operational documents fully describing these steps have not been produced yet. The following tips can be recommended: | ||
*Since Lidar data are extremely voluminous (exceeding 20 Mo per km²), ascii data (xyz) can best be binned into reasonably sized tiles, making processing lighter. These tiles could cover 4 km² for instance, which implies, for a density of 0.5 dot/m², a number of 2*106 dots per tile. Tile organisation is shown in the sketch below and the shapefile of the tiles assemblage should be provided alongside the data. | *Since Lidar data are extremely voluminous (exceeding 20 Mo per km²), ascii data (xyz) can best be binned into reasonably sized tiles, making processing lighter. These tiles could cover 4 km² for instance, which implies, for a density of 0.5 dot/m², a number of 2*106 dots per tile. Tile organisation is shown in the sketch below and the shapefile of the tiles assemblage should be provided alongside the data. | ||
| − | |||
| − | |||
*When data are delivered in individual tiles, care should be taken with edge effects when performing the interpolation. A way to avoid this effect is to have some overlap between the tiles, so that the interpolation will fully apply further than the tile boundary and stitching will be seamless (figure 1). An overlap of say 50 metres would be more than sufficient (see sketch in Figure 1). | *When data are delivered in individual tiles, care should be taken with edge effects when performing the interpolation. A way to avoid this effect is to have some overlap between the tiles, so that the interpolation will fully apply further than the tile boundary and stitching will be seamless (figure 1). An overlap of say 50 metres would be more than sufficient (see sketch in Figure 1). | ||
*Several grid mesh sizes should be readily planned since DTMs are likely to serve various purposes, from very fine to coarser ones. For hydrodynamic modelling, typically 20 metres are more than enough, even for the best inshore refinements. For synoptic rendering, 5 metres would be a convenient mesh size which can be rapidly displayed on any system by less expert users. For specific uses, a very high resolution DTM whose mesh size should mimic the data's average dot density (typically 1 to 2 metres) should be produced. At this resolution, Lidar DTMs can easily be used jointly with scanned aerial imagery. | *Several grid mesh sizes should be readily planned since DTMs are likely to serve various purposes, from very fine to coarser ones. For hydrodynamic modelling, typically 20 metres are more than enough, even for the best inshore refinements. For synoptic rendering, 5 metres would be a convenient mesh size which can be rapidly displayed on any system by less expert users. For specific uses, a very high resolution DTM whose mesh size should mimic the data's average dot density (typically 1 to 2 metres) should be produced. At this resolution, Lidar DTMs can easily be used jointly with scanned aerial imagery. | ||
*In the case where a seamless DTM is produced across the water line, it is necessary to identify and discard the water points on the topographic Lidar. It is recommended that the interface height be identified in a steeper place and that this threshold then be applied to the whole DTM. Assumptions of constant water levels need to be made for the whole area. | *In the case where a seamless DTM is produced across the water line, it is necessary to identify and discard the water points on the topographic Lidar. It is recommended that the interface height be identified in a steeper place and that this threshold then be applied to the whole DTM. Assumptions of constant water levels need to be made for the whole area. | ||
| + | [[image:lidar_fig_07.jpg|thumb|300px|left|Figure 2: Hydrographic Lidar DTM over a coastal area of Côtes d'Armor, Brittany, France]] | ||
| + | ===Example of a hydrographic lidar DTM === | ||
| + | |||
| + | Figure 2 here illustrates a subtidal DTM of northern Brittany made from a lidar survey. The blue line is the LAT (lowest astronomical tide) line and the 10 and 20m depth contours are also shown. As the survey period was between mid and low neap tide, water height is currently between 2 to 5 metres above LAT, which means a total penetration of light beams up to 22 to 25 metres. Lidar dot density is from 3m (nominal) to about 5m in deeper parts, due to extinction of some beams. The DTM was interpolated to a 3m mesh size. | ||
| + | <br style="clear:both;"/> | ||
| − | |||
| − | + | ==Related articles== | |
| + | * [[Use of Lidar for coastal habitat mapping]] | ||
| + | * [[Optical Laser diffraction instruments (LISST)]] | ||
| + | * [[Optical remote sensing]] | ||
| + | * [[General principles of optical and acoustical instruments]] | ||
| + | * [[HyMap: Hyperspectral seafloor mapping and direct bathymetry calculation in littoral zones]] | ||
| + | * [[Satellite-derived nearshore bathymetry]] | ||
| − | |||
| − | |||
| − | |||
| − | |||
==References== | ==References== | ||
<references/> | <references/> | ||
| + | |||
{{author | {{author | ||
| Line 73: | Line 83: | ||
|AuthorFullName=Jacques Populus}} | |AuthorFullName=Jacques Populus}} | ||
| − | + | ||
| − | + | [[Category:Coastal and marine observation and monitoring]] | |
| − | [[Category:Coastal and marine | + | [[Category:Observation of physical parameters]] |
| − | [[Category: | ||
| − | |||
Latest revision as of 12:22, 7 December 2023
After Lidar (Laser Induced Detection And Ranging or LIght Detection And Ranging) has been used, collected data needs to be converted to required outputs. This article describes the processes related to data post-processing, products and a Digital Terrain Model (DTM).
Contents
Data post-processing and products
Data quality check
Processing topographic Lidar data requires two major steps a) an internal quality check of the raw data and b) an external quality check. Quality checking mainly means making sure of three points: the data density (to ensure suitable DTM output) and the horizontal and vertical accuracy of data. Data density may not be complied with when the survey navigation is not properly carried out, resulting in gaps between adjacent swaths or over water patches (since water theoretically absorbs the infra-red radiation). The operators usually provide a density map along with the data files. Quality control for topographic data involves getting rid of outliers which may be a result of some obstructions to the light path. It also means filtering objects of no direct interest for the user (typically vegetation, houses and any human objects) to arrive at a DTM representing true ground heights. While the ground signature can be retrieved fairly easily on topographic Lidar, the bottom signature is much more complex to detect on hydrographic Lidar. In practice, all groups of dots are individually reviewed by the interpreter who decides whether the bottom has been found or not. This also depends on the context (neighbouring dots) and examination of the waveforms. Light backscatter may be modulated by standing vegetation (e.g. sea grass or kelp) which can jeopardize bottom detection.
Typical topographic Lidar products
The first products the Lidar operator is asked to provide are triplets (xyz) expressed in an ellipsoid vertical reference and in any suitable horizontal coordinate system. Table 2 shows the various products. Data sets must have been post-processed into true height data without any outliers. In most cases this should be double checked using appropriate software capable of dealing with large data sets. While a gridded product may be produced by quick interpolation on a medium size mesh (typically 5 metres) for use as a data quick-look, the user may want a ready-for-use DTM grid file in the best possible resolution, e.g. a metric one. In this case, this must be specified at the outset of the work. More advanced prefer to produce their own interpolation (see Table 1).
Typical hydrographic Lidar products
Typical output products from a topographic Lidar survey are listed in Table 2 below. The system actually delivers the position (X,Y,Z) of the bottom with reference to the WGS 84 ellipsoid but also the water height, which in turn gives the water surface level. Surface level is actually derived from a large averaging window which ensures more reliability and also provides wave height. Each dot is flagged with an indication whether land or water was found and in the latter case whether bottom was detected. A timestamp allows to assess the data against tidal data. The amplitude of the signal is also available, however if returning photons go through a photo-multiplicator, the resulting signal is in volts, which makes the energy budget difficult. Some systems provide a software to visualise the waveforms at each dot, which could provide an indication of bottom type (see "Lidar backscatter intensity" below).
Data accuracy
The literature commonly reports horizontal and vertical accuracies of respectively about 0.30 and 0.15 rms. Hydrographic data are usually compatible to the IHO standard one quality, i.e. 5 m horizontal and 0.25 m vertical rms accuracies. This reduced accuracy is due to the larger beam divergence of the green beam which typically creates a 3m diameter footprint at sea surface. Since the footprint at the bottom is at least this size, light producing the backscatter peak is reflected from a large area, which reduces accuracy. Lidar systems also collect a directly downward-looking, geo-referenced video concurrently with the Lidar measurements. In addition to offering a visual record of the survey area, the video is frequently used to position coastal features such as navigation aids, piers, and other objects of interest.
Lidar backscatter intensity
Lidar systems are also capable of capturing intensity data. Intensity is a by-product which is not guaranteed by operators. These data may be processed to produce a geo-referenced raster file which looks somewhat like a conventional image. However these data are not calibrated to a physical quantity and only have an indicative value. An attempt was made to merge topographic Lidar intensity with two visible ortho-photo channels to generate an IRC (infra-red colour) composite. Some adjustments had to be made to balance the intensities of the three channels. Hydrographic Lidar also delivers the intensity of the bottom backscatter which could be built into a bottom grey image in the green channel. However, the procedures to extract signal intensity have not yet been stabilised and some research is still needed before assessing these capabilities.
Data processing to Digital Terrain Models (DTMs)
Data validation
Topographic Lidar accuracy is checked by way of high quality DGPS determination of reference surfaces. In practice, it is recommended to survey two of these references per aircraft sortie, basically one at the start and the other at the end. Typically, these surfaces should be smooth and rather flat, so that the horizontal inaccuracy has a limited influence on vertical accuracy. The best surfaces are playing grounds, with a large flat area surrounded by vertical objects (hedges, railings, posts). Validation is a two-step process. The horizontal positioning check should be carried out first: this is done by surveying a number (e.g. 30) of “vertical objects” in the field, namely their footprints on the ground. Once the horizontal accuracy has been shown to be within the specified limits (i.e. between 0.5 to 1 metre rms for topographic Lidar dots), the vertical check can be performed. A set of surveyed points no further apart than the Lidar dot spacing (i.e. about 3 to 5 metres) is selected. Lidar spots no more than one metre away are then chosen and paired with the ground points. Sufficient numbers of these pairs can then be processed statistically. The literature (Huising and Pereira, 1998[1], Joinville et al. 2002[2], Populus et al , 2003[3]) shows than on a bare, smooth and reasonably sloped terrain, vertical accuracy better than 0.15 m rms is always obtained with topographic Lidar. These figures degrade with the terrain type, e.g. low-lying vegetation such as tidal marshes, and slopes as is the case in cliff type shorelines. With hydrographic Lidar, ground truth is much more complicated to achieve for obvious reasons: being underwater, the co-location of Lidar measurements and field checks is much more difficult. Places with bare sediment bottom and which can be safely accessed by boat must be chosen. Ideally, field data should be recorded with a shallow water multibeam sounder. It is crucial that all measurements be carried out with respect to a vertical system independent of the water levels (ellipsoid heights). Very few reports are available to date on validating bathymetric Lidar with acoustic data.
Producing a DTM
Data interpolation is not dealt with here, since many suitable references can be consulted by readers. Some authors (Joinville et al, 2002[2], Daniels, 2000)[4] give a good account of their procedure and guidance can be found there. However, operational documents fully describing these steps have not been produced yet. The following tips can be recommended:
- Since Lidar data are extremely voluminous (exceeding 20 Mo per km²), ascii data (xyz) can best be binned into reasonably sized tiles, making processing lighter. These tiles could cover 4 km² for instance, which implies, for a density of 0.5 dot/m², a number of 2*106 dots per tile. Tile organisation is shown in the sketch below and the shapefile of the tiles assemblage should be provided alongside the data.
- When data are delivered in individual tiles, care should be taken with edge effects when performing the interpolation. A way to avoid this effect is to have some overlap between the tiles, so that the interpolation will fully apply further than the tile boundary and stitching will be seamless (figure 1). An overlap of say 50 metres would be more than sufficient (see sketch in Figure 1).
- Several grid mesh sizes should be readily planned since DTMs are likely to serve various purposes, from very fine to coarser ones. For hydrodynamic modelling, typically 20 metres are more than enough, even for the best inshore refinements. For synoptic rendering, 5 metres would be a convenient mesh size which can be rapidly displayed on any system by less expert users. For specific uses, a very high resolution DTM whose mesh size should mimic the data's average dot density (typically 1 to 2 metres) should be produced. At this resolution, Lidar DTMs can easily be used jointly with scanned aerial imagery.
- In the case where a seamless DTM is produced across the water line, it is necessary to identify and discard the water points on the topographic Lidar. It is recommended that the interface height be identified in a steeper place and that this threshold then be applied to the whole DTM. Assumptions of constant water levels need to be made for the whole area.
Example of a hydrographic lidar DTM
Figure 2 here illustrates a subtidal DTM of northern Brittany made from a lidar survey. The blue line is the LAT (lowest astronomical tide) line and the 10 and 20m depth contours are also shown. As the survey period was between mid and low neap tide, water height is currently between 2 to 5 metres above LAT, which means a total penetration of light beams up to 22 to 25 metres. Lidar dot density is from 3m (nominal) to about 5m in deeper parts, due to extinction of some beams. The DTM was interpolated to a 3m mesh size.
Related articles
- Use of Lidar for coastal habitat mapping
- Optical Laser diffraction instruments (LISST)
- Optical remote sensing
- General principles of optical and acoustical instruments
- HyMap: Hyperspectral seafloor mapping and direct bathymetry calculation in littoral zones
- Satellite-derived nearshore bathymetry
References
- ↑ Huising E.J., Gomes Pereira L.M., 1998. Errors and accuracy estimates of laser data acquired by various laser scanning systems for topographic applications, ISPRS Journal of Photogrammetry and Remote Sensing 53, 245-261: pp. 17.
- ↑ 2.0 2.1 de Joinville O., Ferrand, B., Roux M., 2002. Levé laser aéroporté : Etat de l'art, traitement des données et comparaison avec des systèmes imageurs, in : Bulletin SFPT N°166, pp 72-81.
- ↑ Populus J., Laurentin A., Rollet C., Vasquez M., Guillaumont B., Bonnot-Courtois C., 2003. Surveying coastal zone topography with airborne remote sensing for benthos mapping, eProceedings of Earsel's GIS "Remote Sensing of the Coastal Zone", Ghent, 2003 June 5-7, 105 117: pp. 13.
- ↑ Daniels, R.C, 2000. Datum Conversion Issues with Lidar Spot Elevation Data, Photogrammetric Engineering & Remote Sensing, Vol. 67, No. 6, pp. 735-740: pp. 17.
Please note that others may also have edited the contents of this article.
|
