IRS Level 2 Processing Concept Status

IRS Level 2 Processing

Concept

Status

Stephen Tjemkes, Jochen Grandell and Xavier Calbet

— Introduction

— Level 2 Processing Concept

— Description of Modules

— Proxy data

— Conclusion

MTG-IRS mission objective

• Primary mission objective is to monitor small scale water vapour structures in support of regional and global NWP • A secondary objective is to support the monitoring of

atmospheric dynamics through e.g. the provision of clear sky wind, global instability,..

• A further objective is to support emerging applications regarding chemical weather and air quality

• Thus primary mission objective calls for accurate

moisture and to a lesser degree temperature profiles

from MTG-IRS observations.

• The development of operational L2 processing

scheme is presented next. How this will be used for

the implementation of an operational scheme is TBD.

L2 Processing Concept

• Input:

– radiometrically and spectrally calibrated spectra with geometrical information appended

– auxilliary data • Processing split into

– Pre-processing to determine the type of scene:

clear/cloudy, dusty, fire, cloud parameter determination. – Quick processing: retrievals using a fast statistical

technique like EOF linear regression, neural networks, etc… – Processing: Physical retrieval (optimal estimation).

– Post-processing: formating, gridding, QA, … • Output

Processing: The Challenge

• Retrievals can be performed for all scenes (though the quality depends on presence of clouds)

• Example iasi L2 (processing 235 channels per spectrum):

– On current system (2CPU IBM power 4 processor): 0.008333 min/spectrum – IRS: >7 106 _{spectra / BRC}

– => 60 000 min to process one BRC

• Note that this is with respect to IBM power 4 Machines (several years old technology). Current state of the art machines (e.g. Power 6) are significant factors faster.

• Still we need to explore all possibilities to keep the L2 processing affordable. So we need to look at

– Implementation:

– Efficient codes (e.g. different RTM) – Parallelisation

– …

– Processing

End-to-End L2 Processing Chain

• Radiative transfer modeling

• Scenes Analysis

RTM

• Need for an efficient radiative transfer module – Scene – Inversion PCRTM OSS RTTOV Community ? + ++ ++ Maintenance ? +

RTTOV vrs OSS: two different approaches

• RTTOV:

– Statistical model for mean transmission – Multiple scattering (TBC)

– Includes Jacobians

– Difficulties wide band imagers • OSS:

– Course resolution line-by-line

– Multiple scattering (Not yet procured) – Jacobians

RTTOV-OSS Comparison

• 5801 diverse profiles • Sea surface emissivity • T, q, O3 from ECMWF • Rest: climatology • RTTOV-9 – GENLN2, HITRAN 2000, CKD2.4 • OSS – LBLRTM V11.3, HITRAN2004, MT_CKD 1.0 • LBLRTM V11.3 as reference (for 500 profiles)

Timing Results (in sec/profile)

Direct only Direct + Jacobians

RTTOV – 9 1.11 11.42

OSS - 0.67

Comparison to IASI

• First OSS

• Second RTIASI

• Averaged results over 500 cases

– Red line mean

Comparison to LBLRTM

• RTTOV SAD were generated using GENLN2

• OSS SAD were generated using LBLRTM

• Difference in performance could be result of

difference in GENLN2 and LBLRTM.

• Single profile

• Red line OSS

Summary

• For hyperspectral applications:

– Performance: OSS = RTTOV/16

– Accuracy: OSS compares favourable to reference

• Not shown: For Imager applications:

– Performance OSS = 2*RTTOV

– Accuracy: OSS compares favourable to reference

• We will use OSS for our development of IRS L2

Expert Note:

• Reduce the number of mono-chromatic calculations

by OSS significantly through the so-called global

training, as opposed to the localised training applied

here (up to factor 10)

• There is room to improve OSS efficiency, by how

much will depend on application (hyperspectral,

imagery)

Pre-processing: Scenes Analysis

• At Day-1 process only cloud free FOV (+ maybe Low

Level clouds)

• Implemented the SCE by Watts & McNally for

AIRS/IASI.

Cloudy - clear radiances (provided by phil

watts)

Measured

-M

odel (K)

Model noise (in 6 μm band)

Pressure ranking: all channels

Cloud emissivity effect

Measured

-M

odel (K)

Clear-channel id 1: Low-pass filter

δBT(j)

LP[δBT(j)] 10-20 ch

Detect ‘gross’

cloud signal (+ or -)

*

• IASI L2 cloud

mask

Quantitative Comparson

IASI L2 Cloud Mask

No data Cloudy Clear

No data 1818 0 0

Cloudy 66 1571 84

Clear 4 105 73

ECMWF cloud detection (ECD)

Conclusion

• Successfully implemented

• Not yet an extensive validation

• Need to apply to MTG-proxy data

• Integrate into end-to-end development chain

• Replace current rtm with OSS

•

CO2 slicing

alternative methods to be considered

– Gives cloud top pressure and cloud fraction

IRS L2 Prototype Processor tests

• IRS L2 Prototype Processor running with:

– IASI real data converted to IRS with IASI2IRS tool

– IRS spectra have the original noise coming from IASI – RTM: RTTOV-9.1+IASI2IRS

– Clear sky over ocean selection scenes as for IASI (threshold test method only)

– Bias corrected

– Optimal estimation

– First guess from EOF retrieval – Background from Chevallier

IRS L2 Inversion: future

• Improve fast retrieval method -> Neural networks? • Introduce faster radiative transfer model -> OSS

• Keep on verifying with real IASI data correct scene classification • Introduce realistic noise into IRS synthetic measurements

• Introduce pseudo-noise diffraction effects into IRS synthetic measurements using proxy data

• Determine minimal set of channels for with Inversion will be applied

• Analyse co-registration errors

• Analyse correlated noise in observations • Analyse spectral calibration errors

• Apply to field experiements (jaivex), proxy data • Compare to independent methods

Other components

• Surface emissivity retrieval

• Improve accuracy through exploitation of the time

domain (e.g. Kalman filter)

Proxy Data

• For end-to-end processing chain

• For technical studies like compression, error budgets

• Source

– IASI

– Synthetic based on models run by

– SSEC

IASI as proxy for IRS

• IASI L1C can be converted into IRS – Select LWIR or MWIR

– Generate Interferogram – De-apodise interferogram – Truncate Interferogram – Convert into spectrum

• Tool to do this is available upon request

IRS proxy data from WRF

• In support of GOES-R H. Huang, T. Greenwald and xx generated two case studies based upon WRF

• Consider here the European simulation

• Hope to get a sample tape soon, to see if we can transfer data using LTO III data-tapes • Data needs to be converted into IRS (and possible other candidate mission) radiances

16-17 August 2006

00-09 UTC 09-15 UTC 15-00 UTC

Temporal Resolution (minute)

Full disk 103 GB 16 TB 3 15 5 15

WRF model output data volume and spatial and temporal resolution for the NCSA MSG simulation.

Domain File Size per Output Time (GB) Total Dataset Size (TB) Spatial Resolution (km)