Use and influence of composite indicators for sustainable development at the EU-level

(1)

HAL Id: hal-02462754

https://hal.archives-ouvertes.fr/hal-02462754

Submitted on 31 Jan 2020

HAL

is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire

HAL

, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Léa Sébastien, Tom Bauler

To cite this version:

Léa Sébastien, Tom Bauler. Use and influence of composite indicators for sustainable development

at the EU-level. Ecological Indicators, Elsevier, 2013, 35, pp.3 - 12. �10.1016/j.ecolind.2013.04.014�.

�hal-02462754�

(2)

EcologicalIndicators35 (2013) 3–12

ContentslistsavailableatScienceDirect

Ecological

Indicators

j o ur na l h o me p a g e :w w w . e l s e v i e r . c o m / l o c a t e / e c o l i n d

Use

and

inﬂuence

of

composite

indicators

for

sustainable

development

at

the

EU-level

Léa

Sébastien

a

,

Tom

Bauler

b,∗

a_Université_Toulouse_II-le_Mirail,_GEODE_Research_Center_{(Environmental}_Geography),_UT2-Le_Mirail,_Maison_de_la_Recherche,_Laboratoire_GEODE,₅_allées

AntonioMachado,31058Cedex9,Toulouse,France

b_Université_Libre_de_Bruxelles,_Institut_de_Gestion_de_{l‘Environnement}_et_{d‘Aménagement}_du_Territoire,_Centre_d‘Etudes_du_{Développement}_Durable,_ULB_–

IGEAT(cp130/02)50,AvenueFDRoosevelt,B-1050Brussels,Belgium

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received23December2011

Receivedinrevisedform29March2013 Accepted20April2013 Keywords: Compositeindicators Policyuse Sustainabledevelopment EuropeanUnion Governance

a

b

s

t

r

a

c

t

Consideringtheon-goingstrivetowardsnew,alternativeindicatorstomeasureoursocietaldevelopment pathways,andthefactthatpolicyindicatorsremainlargelyenigmaticwithregardtotheirpatternsof embeddednessininstitutionaldecision-makingprocesses,itappearsnecessarytoworktowards reduc-ingourlackofunderstandingoftheirinteractionswithpolicy-making.Inthepresentpaper,wefocuson exploringthesignificanceofcompositeindicatorsforpolicymakingintheparticularpolicyenvironment oftheEU-institutions.Ourresearchisunderpinnedbytheconvictionthatsuchindicatorsarenot system-aticallyuseddirectly,buthaveanindirectinfluenceonpolicymakingthatneedstobebetterunderstood. Ouranalyticalframework–inordertoanalysethewaysinwhichcompositeindicatorsenterpolicy pro-cesses–ischaracterisedbythedistinctionbetweenthe‘use’andthe‘influence’ofindicatorsontheone hand,andontheotherhandbetween3typesoffactors:indicatorfactors,policyfactorsanduserfactors. Ourempiricalresultsshowthatwhilemostoftheacademicattentionandpoliticaldebatearound indi-catorshastendedtofocuson‘indicatorfactors’,suchqualityattributesactuallymatteredrelativelylittle inoursettingasdeterminantsofindicatorinfluence.Thisrejectstheideathattherobustnessofevidence wouldlieexclusivelyinitstechnicalqualityandintheindependenceofitsproducer,andinsteadcalls attentiontotheprocessesofevidence-construction.Simultaneously,‘userfactors’(beliefsand represen-tationsofpolicyactors)and‘policyfactors’(institutionalcontext)werecrucialasexplanatoryfactorsof thepolicymechanicsweidentified.

1. Introduction

Weliveinaperiodofindicatorabundanceinvirtuallyallpolicy domains,andwithanintensifyingpublicandexpert-baseddebate (Stiglitzetal.,2010;OECD,2011;CEC-COM,2009;FAIR,2008)on thechoice,conﬁgurationanddevelopmentofindicatorstomeasure oursocieties’development.Nevertheless,policyindicatorsremain largelyenigmaticwithregardtotheirpatternsofembeddednessin institutionaldecision-makingprocesses(Innes,1998;Ayres,2000; Gudmundsson, 2003; Lehtonen,2004; Hezriand Dovers,2006; Rosenström,2006;Boulanger,2007;Rydin,2007;Turnhoutetal., 2007;Bauler,2012).

Considering the on-going strive towards new indicators,

reducing our lack of understanding of their interactions with

∗Correspondingauthor.Tel.:+3226504924.

E-mailaddresses:[email protected](L.Sébastien),[email protected]

(T.Bauler).

policy-makingappearsnecessary.Westillknowlittleabouthow andwhypolicymakers,stakeholdersandcitizensuseindicatorsin general,andcompositeindicatorsinparticularand,more impor-tantly,whethertheindicatorsactuallyinﬂuencepolicyandsociety atlarge.Inthepresentpaper,wefocusonexploringthesigniﬁcance ofcompositeindicatorsforpolicymaking.Theseparticular indi-catorsaggregatediverseinformationonasectorarea,andresult

inﬁneinprovidingthebasisforitsranking,whichinturnoften allowstocomparetheoutcomesofpublicpolicies.Composite indi-catorsarerepeatedlydescribedasbeingstraightforwardtouseas signal-makers.Because oftheirapparent‘simplistic’ designand

theunambiguousmessagetheyadvance,compositesaremeantto

impactonthespeciﬁcfringeofhigh-levelpolicy-makersaswell asonthegeneralpublic/stakeholders.Theseindicatorshavethe advantageofbeingencompassing,butthedisadvantageofbeing

difﬁcult to decrypt and analyse, because they highly condense

messages,i.e.reality.Ifthe‘offer’ofcompositeindicatorsinthe sus-tainabledevelopmentdomain(SDI-C)issubstantial,i.e.thesheer numberofSDI-Cwhicharedevelopedandpublished,ourgeneric

(3)

researchquestionis toattempttoqualifythe‘demand’,i.e.the receptionofthemessagesbythosepolicyactorswhoactwithin thepolicyarenas(SebastienandBauler,2010).

What’s thesigniﬁcanceofcompositeindicatorsfor

sustaina-bility (SDI-C) at EU-level? We aim to characterise the uses

(includingprocessesof‘non-use’or‘misuse’)andinfluenceof SDI-Cbydecision-makersattheEU-levelandtounderstandthesocial networkinwhichtheseindicatorsareembedded.Empiricallythe focusoncompositeindicatorshasadoubleadvantageover disag-gregatedindicatorsorindicatorlists:compositeindicatorshaveat leastonerelativelycleartarget-usergroup(i.e.high-levelpolicy makers)andarelativelyspecificfunction(i.e.todrawattention toapolicyproblem). Theseinherent characteristicsof compos-iteindicatorshelptocircumscribetheboardersof anempirical explorationofhowcompositeindicatorslinktopolicy.Ourfieldof explorationisfurtherdelimitedbythepolicydomaintowhichthe compositeindicatorsweexplorebelong;weexclusivelyinvestigate thelinkagetothe‘sustainabledevelopment’policyagendaatthe Europeandecision-makinglevel.Thiscasestudyhasbeenstudied inthethree-yearEU-fundedresearchprojectPOINT(policy influ-enceofindicators),whichexaminedtherolesthatenvironmental andsustainabilityindicatorsplay(orindeedfailtoplay)in policy-makingindifferentcountries,domainsandatvariousscales.Our researchisunderpinnedbytheconvictionthatindicatorsarenot systematicallyuseddirectlybuthaveanindirectinfluenceonpolicy makingthatneedstobedeeplystudied.Ourempiricalexploration oftheuseandinfluenceofcompositeindicatorsinpolicy-processes intendstoelucidate themechanicswhich builtuparound indi-catorsasbeingpartofgovernanceprocesses,and,the‘politicsof policyindicators’(Bauler,2012).

Section2belowpresentsthetheoreticalcontext,followedby ourframeworkandmethodologyinSection3.Sections4and5are devotedtoourresultsanddiscussions.

2. Theoreticalcontextualisation

Indicators have been deﬁned as “variables that summarise

orotherwisesimplifyrelevantinformation,makevisibleor

per-ceptible phenomena of interest, and quantify, measure, and

communicate relevant information” (Gallopin, 1996, 108). As a speciﬁcformof operationalisingthe conceptof evidence-based policy,indicatorsareseentoservemultiplefunctions,in partic-ularthoseofcommunicationandawareness-raising(Rosenström andLyytimäki,2006,33),providingearlywarningfunctions, polit-icaladvocacy,andimprovingthequalityofdecisions.Whichever theprimaryobjective,indicatorsareexpectedtoreduce ambigu-ity.Asknowledge-agents,indicatorsareexpectedtocommunicate evidence in a form suited for policy actors by simplifyingthe descriptionofcomplexsystems,eitherthroughreducingthe num-berofkeyvariablesobserved(McCoolandStankey,2004,295)1

orbybuildingclear,unambiguousandwell-deﬁnedvisionsofthe desiredfuture(HardiandZdan,1997).

Theseexpectationsofindicatorfunctionsarehabituallyrooted ina rationalistic and linear conception of theinstrumental role playedbyknowledgeindecision-making,entailingtheassumption that more efﬁcient policy outcomeswill result if robust,

data-driven, presumably objective and value-free evidence is made

availableforpolicymakers,inasimpliﬁedandsynthesisedformat. Byclarifyingissuesandreducingscientiﬁcuncertainties,indicators wouldhenceprovideasharedfactualbasisforpolicydiscussions, andenhance therationalityofdecision-makingand thequality

1_McCool_and_Stankey_(2004,₂₉₅₎_argue_that_“a_basic_purpose_of_indicators_is_to

reducecomplex,poorlyunderstoodsystemstoalimitednumberofvariablesthat presageimpendingchangesinlifesupportandmanagementsystems.”

ofdecisions2_._The_underlying_logic_can_therefore_be_described_as

oneofbetterinformation(becauserobustandsimple)forbetter decisions(becauseeffectiveandefficient).Inthisperspective, indi-catorsarefrequentlyconceivedofasconsensus-buildingtools– essential instrumentsinattempts topacifyconflictand domes-ticatecontroversy,which areseenastime-consumingobstacles toefficientpolicymaking.Suchdomesticationwouldbeachieved throughtheestablishmentofnorms,standards,criteriaand proce-duralrulesformonitoringandregulation(Chateauraynaud,2011, 195). The linear model that sees indicators exclusivelyin such aninstrumentalrolestillprevailsasthemainnarrativeinpolicy practice(Owensetal.,2004).However,researchonknowledgeuse (Vedung,1997;Weiss,1999)hasdemonstratedthatknowledge sel-domplayssuchastraightforwardroleinpolicymaking,butfarmore ofteninfluencespoliciesthroughindirectandlargelyunforeseen pathways,graduallyshapingframeworksofthoughtorservingas ammunitionin politicalbattles.Incontrastwiththedominance ofrationalisticapproachestotheanalysisofindicators,ourpaper drawsattentiontothevariousindirectwaysinwhichindicators cometoexerttheirinfluence.

3. Ananalyticalframeworkfortheexplorationofuseand inﬂuenceofSDI-Cs

3.1. Distinguishingbetweenuseandinﬂuence

Thefirstlevelofouranalyticalframeworktoanalysethewaysin whichindicatorsenterpolicyprocessesisthedistinctionbetween the‘use’and the‘influence’ofindicators.Theseparationofuse andinfluencedrawsonevaluationresearch,notablyonMarkand Henry(2003),whoarguethatwhileindicatorusedoesnot auto-matically leadtoimpact,likewise anindicatorcan significantly influencepolicyandsocietywithoutanyoneactuallyconsciously ‘using’theindicator.Wedefineuseastheconcreteactof hand-lingtheindicatorsinapolicycontext,whereasinfluenceoccurs throughdialogueandargumentation(Valovirta,2002)–either dur-ingtheprocessofelaborationofindicatorsorsubsequenttotheir release.While‘use’tracesbacktheoriginalobjectives,intentions andstrategiespursuedbyactorshandlingtheindicator,‘influence’ enablesonetoidentifythewaysinwhichindicatorsinteractwith policymaking.Influenceisthereforeclosetotheconceptofportée

(inenglish:‘reach’or‘extension’),whichChateauraynaud(2011)

employstoqualifytherole,status,andtheactualstrengthofan argumentinapubliccontroversy.Indicator‘influence’/portéecan concernthetargetedpolicyorbroaderprocessesinsociety,such asadministrativestructuresortheoperationofdemocratic insti-tutions.It canentailneworreconfirmeddecisionsand actions, sharedunderstandings,networkingamongpolicyactorsorchanges inthelegitimacyofpolicyactors(MarkandHenry,2003;Lehtonen, 2005,173).Often,however,thetypes ofinfluenceof indicators mightbeverydifferentfromtheintendeduse.Forinstance, indi-catorsmightbedevelopedwiththeobjective ofmonitoringthe evolutionofa particularvariable(say,theshare ofsolarpower innationalenergyproduction)deemedtoprovideinformationon policyeffectiveness (inpromoting theproduction ofrenewable energy).However,thepublicationanduseofthe‘solar power’-indicatorsmighttriggeramuchwiderthanintendeddiscussionon

2_Examples_of_such_perception_of_indicators_as_objective_measurement_tools_are

easytoﬁndintheliterature.ThedevelopersoftheEnvironmentalSustainability Index(ESI)arguethat“[t]heESImovesustowardsamoreanalyticallyrigorous anddatadrivenapproachtoenvironmentaldecisionmaking”(DevittandDeFusco, 2002),whiletheorganisation“RedeﬁningProgress”(2003)declaredthatitsought to“measuretherealstateofoureconomy,ourenvironment,andsocialjusticewith toolsliketheGenuineProgressIndicatorandtheEcologicalFootprint”.

(4)

L.Sébastien,T.Bauler/EcologicalIndicators35 (2013) 3–12 5 theenvironmentalimpactsoftheproductionandrecyclingofsolar

panels,andhenceraisequestionsabouttheirpublicacceptance– anoutcomeintermsoftheinfluenceoftheindicatorswhichisat leastpartlycontrarytotheobjectivesofthesolarpoweradvocates. Summingup,‘influence’enablestheidentificationofthewaysin whichindicatorsinteractwithpolicymaking,whereas‘use’traces backtheobjectivesandintentionspursuedbyactorshandlingthe indicator.

Useandinﬂuencehavetheirinstrumentaldimensions(asseen earlier)but, more importantlyfor ourpaper,alsotheir concep-tualand politicaldimensions.Conceptualuseandinﬂuenceoccur whenindicatorshelptoconstituteabroadinformationbasefor decisionsbyshapingconceptualframeworks,mostlythrough dia-logue,publicdebate,andargumentation.Anindicator,orasetof indicators,henceprovidesgeneralbackgroundinformation, lead-ingto“enlightenment”amongpolicyactors,i.e.“thepercolationof newinformation,ideasandperspectivesintothearenasinwhich decisions are made” (Weiss, 1999),and sharedunderstandings (Innes,1998).Inthisway,indicatorswouldaffectdecision-makers’

and other policy actors’ problem deﬁnitions, and provide new

perspectives and insights, rather than targeted informationfor a speciﬁc point in decision-making, or toa hypothetical single decision-maker.Giventhecomplexityanduncertaintyaswellas thediversityofnormativestandpointsinvolvedinpolicies concern-ingsustainabledevelopment,itcanbearguedthatsustainability indicatorsmostoftenprovidesharedconceptsandopenuparange ofalternativeperspectives,ratherthanproduceconclusive assess-mentsofspeciﬁcpolicyoptions.

Lastly,indicatorsarefrequentlysubjecttoandgenerateovertly

politicaluseandinﬂuence,helpingtojustifyorlegitimisepolicies, decisionsand actorsorpostponedecisions.Ortega-Cerdà(2005)

mentionsthesymbolicfunctionoftheprocessesofknowledge cre-ationassubstitutesforactionorasadelayingtactic.Knowledgeis thenusedtojustifypre-determinedpositions,inordertopersuade politicians,civilservants,stakeholdersorpublicatlargetypically

aboutpolicy plansthat are already wellontheir way towards

implementation(HezriandDovers,2006;Patton,1997).Indicators wouldhenceconstituteafaçadeintendedtogivetheimpressionof arationalorganisationthatsetsgoals,ispreparedforchange,hasa seriousandcompetentmanagement,andtakesrationaldecisions onthebasisofdatawhichisitabouttoconstruct(Pollitt,2006).In extremecases,indicatorscanbe‘misused’bydistortionorthe omis-sionofsignificantelements(Weissetal.,2005).However,giventhe essentialroleofjustificationandpersuasioninpolitical decision-making(Valovirta,2002),farfrommereillegitimatemanipulation,

politicaluseandinﬂuencecanalsoconstituteanessentialsourceof democraticlegitimacy,oraformofadvocacyforsocially progres-siveobjectives,suchassustainabledevelopment(ParrisandKates, 2003;Holden,2008).

3.2. Identifyingexplanatoryfactors

Thesecondlevelofouranalyticalframeworkstrivestoexplain theobserveduseandinﬂuencepatternsthroughthreeexplanatory factors,following Pregernig (2000),who suggested that knowl-edgeuptakeisgovernedbythreefactors:(a)theinformationitself (whatwecallthe‘indicatorfactors’,relatedtothevariousquality attributesoftheindicator),(b)theknowledgeandexperienceof thereceiver(here,the‘userfactors’,notablythe‘repertoires’(van derMeer,1999)oftheactorsinvolved),and(c)theexternalsettings (‘policyfactors’,denotingthegeneralpolicycontext).

‘Indicatorfactors’includethequalityandstructuringofdata, selectionofthevariables,weightingschemes,timeliness, robust-nessofthemethodology,availabilityofaccurateandappropriate datasources,methodsofcommunication,construction, aggrega-tion and presentationof indicators. The criteria relating to the

quality of indicators include the conventional determinants of scientiﬁc data: validity, reliability, speciﬁcity,and sensitivity to changesinthefactorofinterest.Theunderlyingassumptionhere isthatwheninformationis“relevant,timelyandcomprehensible, itwillbeused”(Romsdahl,2005).

‘Userfactors’describetherepertoires–aconceptthat encom-passesactorexpectations,beliefsystems(Sabatier,1991),mental modelsandthemoreoperationalcodesandpracticesof organi-sations.Repertoirescanbeconceptualisedas“stabilisedwaysof thinkingand acting(ontheindividuallevel)orstabilisedcodes, operationsandtechnology(onotherlevels)”(vanderMeer,1999, 390). Attheindividuallevel, anactorpositions himself person-ally in relation to an indicatorafter having read, digested and interpreted it (Hezri and Dovers, 2006); at thecollective level, repertoiresrelateinparticulartothedegreetowhichtheuseof indicatorsispartofanorganisation’soperationalroutines.Acrucial elementofuserfactorsrelatestotheimplicitorexplicitconceptual modelunderpinninganindicator:towhatextentdoesthewayin whichanindicatorframestherealityandtheproblemsinquestion

correspondtotheconceptualframeworkoftheusersandother

actorsinvolved?Repertoiresareimportantinthattheygovernthe

choicesthatusersmakeamongthemultitudeofinformationon

offer.

Finally,thetypeanddegreeofuseandinfluenceofindicatorsis affectedbythegeneralpoliticalcontext inwhichtheindicators arebeingdeveloped andused.Such‘policyfactors’concernthe meta-settingwithinwhichindicatorsarebeingusedandproduced, andinclude,inparticular,thelong-termframeworkconditions,the short-termshifts ingoverningcoalitions andthecharacteristics of thepolicyissues atstake.Animportantaspectofthe‘policy factors’ is theinstitutionalisation of indicators, that is, the set-ting upof proceduresand practices that allowfor anindicator tobecomeareferenceinpolicyprocesses,andwhichlegitimises andformalisesthemethodsandconceptsunderpinningthe indi-cator(Innes,1998).Thedegreeandtypeofuseandinfluencethat an indicatorengendersare shaped by itsrelevance for specific policyobjectivesandpolicyprocesses.FollowingKingdon(1984),

one couldhypothesisethat theuse andinﬂuence of an

indica-torwouldbecompromisedifthepolicyproblemsitdescribesdo notadequatelymatchwiththeprevalentpolicyideasontheone side, anddo notface afavourablepolitical environmentonthe other.

Thesethreesetsoffactorsrelatetothreedistinct,while interde-pendent,levelsofpossibleanalysis.Policyfactorsdependlargely onthepolicydomain,userfactorsrelatetothepolicyactors’ per-ceptions,whileindicatorfactorsrefertothenatureoftheobject, i.e.theindicator.Thesethreelevelsformthebasisforouranalysis oftheuseandinﬂuenceofSDI-Cinpolicymaking.

Tosumup(seeFig.1below),thedegreetowhich an indica-torisusedand/orinﬂuencespolicyinstrumentally,conceptuallyor politicallydependstovaryingdegreesonindicator,userandpolicy factors.Ourobjectivehereistodepicttheroleofeach explana-toryfactorin thevarious typesofpolicyusesandinﬂuencesof SDI-C.

3.3. Empiricalmethodology:aqualitativeapproachtoexplore indicatoruseandinﬂuence

Wefocusedourempirical explorationona setof12 SDI-Cs, eachofwhichcanbeclassiﬁedintooneofthethreedimensions ofsustainabledevelopment:

•Social sphere: main focus on the Human Development Index (HDI),additionalattentionwasgiventotheHappyPlanetIndex,

(5)

Use and Inluence

Instrumental Conceptual Political

Indicator factors

User f

actors

Policy factors

Fig.1.Analyticalframework.

•Environmental sphere:main focusonEcologicalFootprint(EF), additional attention toLiving PlanetIndex, Ecosystems Well-beingIndex,EnvironmentalPerformanceIndex.

•Economicsphere:mainfocusonGenuineSavings,(GS)additional attentiontoIndexofSustainableEconomicWelfare,Corruption PerceptionIndex,GiniIndex.

The three headline SDI-Cs (HDI, EF, GS) wereprioritised by

the EU-level ‘Beyond GDP’-process (today relabelled as ‘GDP

andbeyond’) asimportantfeaturing indicators3_._Empirically, _we

deployed3perspectivestoourqualitativeanalysis:anindividual level(viaface-to-faceinterviews);aninterpersonallevel(usinga ‘participant-observer’approach),acollectivelevel(viadocument analyses).

InordertodepicttheuseandinﬂuenceofSDI-Catthe individ-uallevel,anobviouslyimportantperspectivetobeexploredwhen tryingtograsptheperceptionofpolicy-makerswithregardto indi-cators,weconducted24semi-structuredinterviewswithhigh-to

mid-levelpolicy-makersfromtheEuropeanEnvironmentAgency,

DGEnvironment,DGResearch,DGEducation,DGInternalmarket

andEUROSTAT.Theinterviewguidedevelopedaroundthe follow-ingthemes:decision-andpolicymakingprocesses;sustainable developmentand well-beingindicators;thepertinenceof indi-catoraggregation;theconﬁgurationofanidealSDI-C;thepolicy targets,policyobjectivesandpertinentscaleofSDI-C;thesocial networkaroundSDI-C;theinterest,concernandtrusttowards SDI-C;the‘demand’-characterisationforSDI-C;thelegitimacycriteria forSDI-C.

In parallel to the face-to-face interviews, the participant-observer approach has been implemented in a series of pre-identiﬁedworkshopsand(international)conferencesspeciﬁcally dedicatedtodebatetheissueofEU-levelindicatorsin2009and 20104_._The_objective_was_to_observe_{interactions,}_dynamics,

dis-coursesandrepresentationsofpolicyactorsattheseevents,with

theaim to grasp what sort of messages policy actors publicly

defend,argue,anddebateonthematterofSDI-C.Ourempirical settingenabledustodirectlyobserve-viaourfocusonSDI-C -thedisputeofinterpretationswithinthepolicyarenaofSD(i.e. whatisthe‘vision’ofSDatEU-level?),andtoindirectlyobserve thedisputebetweenpolicyarenas(i.e.wheredoesSDstandwith regardtoother‘visions’oftheEuropeandevelopmentpathway?). Weareshiftingfromtheindividuallevelofperceptions(whichwas thefocusofourface-to-faceexplorations)towardsthe interper-sonallevelofthesharedunderstandingoftheuseandinﬂuenceof SDI-Cswithinpolicyprocesses,e.g.theroleofSDI-Csinthe pro-cessesofargumentationanddialogueamongactors(persuasion, legitimisation,criticism...)(Valovirta,2002).

3_{http://www.beyond-gdp.eu/indicators.html}_.

4_Such _as _the _Sustainable _Development _Indicators _working _group _within

EUROSTAT(Luxemburg,2009–2010);theOECDWorldforum(Korea,2009);the conferenceonBeyondGDPandEcologicalfootprint(Brussels,2009);the EPIGOV-project’s (Environmental Policy Integration and multi-level governance) ﬁnal conference(Brussels,2009).

Athirdlevelofanalysiswasimplementedthroughdocument

analysisofformal,strategicpolicydocuments5_._These_documents

speakforthecollectivelevel,i.e.theyarecollectively,politicallyor scientificallyacceptedpolicydocumentswithaninfluenceonthe configurationofpolicyarenas.ThepercolationofSDI-Cwastraced throughout theentirepolicyprocesses byconcentrating onthe identificationofindicator-relatedcontentwithinthesepolicy doc-uments.Thecollectivelevelexaminesoutcomesintermsofpolicy decisionsand actions,sharedbeliefs andunderstandings, legiti-macyofpoliciesandactors,agenda-settingandnetworkformation (MarkandHenry,2003).

Whenexploringouranalyticalframework,whichcollates3 fac-torsvs.use/inﬂuenceofSDI-Cs,throughthese3differentlensesof thepersonal,interpersonalandcollectivelevels,wewereaiming todepicttheEU-levelpolicy-makers’perceptions(personallevel), theirdebates,discoursesandconﬂicts(interpersonallevel)andthe formaloutcomesofthepolicyprocesses(collectivelevelvia docu-mentanalyses).

Inthefollowing,ourresultsareorganisedintwoparts.First, wedevelopourresultsoverthethreelevelsofanalysis(individual, interpersonal,collective).Subsequently,wediscusstherolesofthe threedifferentidentiﬁedexplanatoryfactorsofindicatoruseand inﬂuence(i.e.user,indicator,policyfactors).

4. Results(part1)–fromperceptionstopolicyoutcomes: littleevidenceofindicatoruse

4.1. Individuallevel:identifying3confusions

Theindividualleveldevelopedintoanimportantlevelof read-ingwhenitcomestointerpretingtheuseofindicators.Intheend, itisindividualswhoaremeanttooperate,i.e.touse,the indica-torsintheirdailyliveaspolicy-makers.Thefactthatmostofthe

interviewedpolicy-makershadnomemoryofhaving“used”

SDI-Csintheirprofessionalsettingappearedthroughtheinterviewsto originatefrom3confusions.

First, mostofourEU-level policy-makershad a poorfactual knowledgeofSDI-Csingeneral.Theinterviewedpolicy-actors gen-erallyhadsomebasicknowledgeonlyoftheEcologicalFootprint (EF)andtheHumanDevelopmentIndex(HDI).Theyoftendeclined toexpresstheiropinionsevenontheseSDI-Cs,arguingthatthey werenotspecialistsofthesustainabledevelopmentpolicyagenda. Nevertheless,mostoftheintervieweesexplicitlystatedthatthey wereinterestedandactivelyfollowedtheEuropean-level“Beyond GDP”-process;thissituationresultsfromaconfusionbythe inter-vieweestoadequatelyseparateapolicyprocessonchangingthe indicatorsfromapolicyprocessonchangingthesocio-economic characteristicsofoursocieties(i.e.“BeyondGrowth”).Thelackof factualknowledgeaswellasofawarenessofthenatureoftheSDI-C relatedpolicy-processcansimplybeasignofinformation

over-5_Lisbon_strategy_indicators;_EU_Sustainable_Development_Strategy_(SDS)

indica-tors;DGEnvironmentindicators;EuorpeanEnvironmentalAgency(EEA)indicators; GDPandbeyondindicators;EU2020indicators;theStiglitz-Sen-Fitoussireport.

(6)

L.Sébastien,T.Bauler/EcologicalIndicators35 (2013) 3–12 7

ﬂow:toomanyprocesseslinkedtotoomanyindicators.On the

otherhand,itisevidentfromtheinterviewsthat–atleastpartly –powerrelationsandvestedinterestsmayinducepolicy-makers tostatenon-existingignoranceandconfusion(SagerandRavlum, 2005;Flyvbjerg,1998).

Second,theinterviewedpolicy-makersdidnotfeeldirectly con-cernedbycompositesustainabilityindicators.Again,thisappears partlysurprisingbecauseSDI-Csarespecificallydesignedforboth policy-makersandsociety atlargebyaimingtoprovideaquick message to non-specialists. Indicators of a more disaggregated naturewereclaimed tobemoreinfluential thanSDI-Cs.Atthis level,thenon-useofSDI-Cscanbeexplainedbythefactthat policy-makersandthemoregeneralpubliceffectivelyhavedifferentneeds intermsofindicators;thattheyshouldnotbeconfusedasan iden-ticaltargetgroup.TailoringSDI-Csforoneortheothercouldbe essential.Indeed,theindicatorproducerswereoftenseenasbeing tooignorantofthespecificitiesofthetargetgroup(i.e.the policy-makers),beingingeneraldisinterestedinthewayinwhichtheir indicatorswerereceivedbythetargetedusersandaboutthetype ofusethattheirindicator(s)mightengender.

Athird explanationfortheminor(instrumental)useof

SDI-Csisgroundedintheacknowledgementbyallourinterviewees

thatSDI-Cscanbeconfusingforbeingatthesametimegeneral ‘worldvisions’andtargetedevidenceforpolicymaking.Our inter-vieweesarguedthatwhileindicatorproducersseekinprincipleto matchinformation-provisionwithpolicy-makingneedsinorderto allowtimelyandappropriatedecision-support,SDI-Cswere advo-catingacertain‘worldvision’(e.g.theﬁnitenessofenvironmental resources),whichisnecessarilypartisanandparticulartoa spe-ciﬁcindicatorproducer.Thepolicy-makersstronglyunderlinedthe inherentlysubjectivenatureofindicators,arguingthatjustabout everystepinindicatorproductionanddiffusionisarbitrary,a “mat-terofchoicemadebyoneparticularactoratoneparticulartime fornoparticularreason”6_._They_argued_that_such_subjectivity_and

arbitrarinessisevenstrongerforcomplexandpoorlydeﬁnedpolicy issuessuchassustainability,whichremainssubjecttonumerous rivalattemptsofdeﬁnition.

4.2. Interpersonallevel:evidentconﬂictsbetweenpolicy-makers

TheEuropeanUnion’sinstitutionsarearatherunifyingpolicy structure;Directorate-Generals, EuropeanParliament, European

Council, European Agencies co-develop policies and the more

importantofthemaredecidedonacommonbasis.However,during policydevelopmentandconfiguration–butalsowhenitcomesto justifyingpasseddecisions–debatedoesofcourseprevailamong policymakers,anditappearsintheseprocessesthatourtarget indi-catorsare‘dis-unifying’tools.SDI-Csstirthepolicydebatesrather thanhelpingtosettlethem.Theparticipant-observationofaseries ofhigh-tomid-levelpolicyevents(e.g.conferences,workshops, workinggroups) allowedtoidentifythenatureof theconflicts betweenpolicy-makersthatarefoughtwiththehelpofSDI-Cs.This discourse-enhancingcharacteristicofSDI-Cscanbeshownthrough thefactthatarelativelyunambiguouscompetitionexistsbetween thevariousindicatorsasproposedbyspecificDGs,Institutionsand Agencies.Theobservationsrevealedimportantinter-institutional conflictsaboutchoice,supportandadequacyofpotentialSDI-Cs, toanextentwhereindividualEU-policy-makersappeartobeless inoppositionwithother,non-institutionalstakeholdergroups(e.g. NGOs)thanwiththeirdirectcolleaguesfromwithinEUinstitutions.

When looking closely, the conﬂicting debate on SDI-C

mir-rors the fact that individuals in different EU-institutions carry

6 _Quote_of_an_interviewee.

differentiatedrepresentationsaboutthemeaning ofsustainable development.Weiss(1999)andLeknes(2001)purportthatthe pol-icyinfluenceofindicatorsislikelytobegreaterinconsensualpolicy domainsof‘professional’character,aswellasonpolicyissuesthat havereachedastatusofhighinstitutionalisation.Hence,despite thefactthattheEU-levelhasinstitutionalised‘Sustainable Devel-opment’(SD)asoneofitsoverarchingpolicygoalsthroughthe developmentoftheEU-SustainableDevelopmentStrategies(2001 and2006),policyuseofSDI-Csmightwellbehamperedbecauseof thelackofconsensusonthemeaningandcontentofSDasapolicy ideaorpolicydomain.Oneoftheallegedbenefitsofindicatorsis theirabilitytoprovidefocusandtoenhanceshared understand-ing, for example around thegoals of anorganisation (Van Der Knaap,2006).InthecaseofSDI-Cs attheEU-level, this mecha-nismdoesnotappeartounfoldunambiguously;thepolicyarena oftheEU-SustainableDevelopmentStrategiesseemstoallowthe confrontationofdifferent‘worldvisions’,wheretheextentofuse ofSDI-Csappearstodependmainlyontheirpotentialfor enhanc-ingconflicts.Oneoftheproceduralconsequencesintermsofpolicy outcomesisthedifficultyofEUinstitutionstoproposeandagree onacommonsetofcompositefeaturingindicators.

4.3. Collectivelevel:lackofappearanceforlackofcoordination

The document analyses unambiguously showed that SDI-Cs

are veryrarelyoccurringeven in thestrategic, relativelyvague

documentswhichwereproducedfortheprocessaroundthe

EU-SDS.Quitedirectly,thisabsenceofSDI-Cscansimplyberouted in thefact that “thereis currently no comprehensive

environ-mental indicator that can be used in policy debates alongside

GDP”(CEC-COM,2009:433),andthatSDI-Csarequitepopularly shelvedintheenvironmentaldomain.Aseriesofinter-institutional debatesovermethodologicalissuesaroundSDI-C,aswellasaround thepotentialpolicyacceptanceofcompositeindicatorsperse,is implicitlyapparentinthescreenedEU-leveldocuments.Butthe EU-Commissionservices(aheadofwhichEurostat)continueto pre-ferawidelistofnon-aggregatedindicators,complementedbyalist ofheadlineindicators.

Toacertainextent,EU-institutionsrecognisetheexistenceand potentialof SDI-Cs, butwithoutusing themdirectlyas a refer-encepointintheirpublications.Thedocumentanalysesrevealnot

muchmoreevidenceonthedebatearoundthe3targetedmain

SDI-CsthanstatingthatUNDPisattheoriginofthe‘HDI’,thatthe WorldBankcalculated‘GenuineSavings’,andthatseveralNGOs developedthe‘EcologicalFootprint’(CEC-COM,2009:433).Other relativelyobviousSDI-Ccandidates,suchasthe‘Indexof sustain-ableeconomicwelfare’(ISEW),the‘Livingplanetindex’(LPI)orthe ‘Humanwellbeingindex’(HWI),donotappearatallinofﬁcialEU documents.

WhileSDI-CarethuswidelyignoredinstrategicEU-documents, thewiderpolicyarenaisdedicatedtotherenewalofindicators, i.e.statesitsengagementtoreplacingorcomplementingGDPand usingalternativeindicators.TheEU-institutionsruntheir‘Beyond GDP/GDPandbeyond’process,cooperatewithOECDonthe redeﬁni-tionofthe‘Measureofsocialprogress’,andexploremethodological avenuessuchasecosystemservices,materialandsocialaccounting. TheStiglitz-Sen-Fitoussireport(2010)hassupportedthedebate withintheEuropeanindicators’landscape,developingsimilar

con-clusionsastheCommission’s“BeyondGDP”-communication:“GDP

doesnotmeasureenvironmentalsustainabilityorsocialinclusion andtheselimitationsneedtobetakenintoaccountwhenusingit inpolicyanalysisanddebates.Theoverallaimistodevelopmore inclusiveindicatorsthatprovideamorereliableknowledgebase forbetterpublicdebateandpolicymaking”(CEC-COM,2009:433). ThelackofuseofSDI-Cscanalsoberelatedtothelackof effec-tivecoordinationandharmonisationofpolicyprocessesacrossthe

(7)

variousEU-institutionsintheirspeciﬁcindicatorwork,andwith regardtoSDI-Csinparticular.ThemessagefromtheEUon sus-tainabilityappearsdiffuseandunclear,anddifferentindicatorsets, basedonpartiallycontradictingconceptualisationsofsustainable development,areproposedbydifferentEUinstitutions7_._As_a_result,

theEU policyonSDI-Cs remainscharacterised bymultiple pol-icystreamsthatseemtohavelittlecontactwitheachother(JRC, 2010).Eventhoughindividualindicatorframeworkswere elabo-ratedseparately,i.e.intra-institutionally,manyofthehighlighted policy-themesoverlap.Asaresult,manyindicatorsarecommonto differentframeworks.EUpolicymakersarepotentiallyconfronted withindicatorsthatcovermorethan30policy-themesandwith morethan250indicatorsthataremeanttoassistthemtoassess policies.

5. Results(part2)–discussinginﬂuenceagainstuser-, policy-andindicator-factors

5.1. Userfactors:explainingtheconceptualinﬂuenceofSDI-Cs

Fromtheabove,itoccursthusthatSDI-Csarehardlyused instru-mentally.SDI-Csareneverthelessidentifiabletobeattheorigin ofindirect,conceptual influenceon policy-makers.Our empiri-calexplorations revealthat SDI-Cs at EU-level tend tobeused asammunition bypolicy actors,notably during theireffortsto legitimisespecificpoliticalandsocietalpositions,andin particu-larwhenarguingfororagainstspecificvisionsandinterpretations ofsustainability.SDI-Csaretoolswhichhelptoshapeandshare visions,andcan thereforebelabelled ‘framework indicators’or ‘framingindicators’,whoseaimistoshapeframeworksofthought, mobiliseaction,aswellastoshapeandgenerateawarenessand symbolicimages (Weiss, 1999; Amara et al., 2004).Innes and Booher(2000)purportthat“indicatorsdonotdrivepolicy.People arenotsuddenlyconvertedbecausetheyareconfrontedwithdata, nomatterhowexpertlyorhowcollaborativelydesigned. Compen-diaofindicatorsarenotusedbypolicymakersasaidstodecision”. Theirgenericfindingappearstobeapplicabletothespecificcase ofSDI-CsattheEU-level.Theimplemented3-foldempirical explo-rationsshowthatSDI-Cscouldshapepolicymakers’worldviews, andtherebypotentiallyinfluencefuturedecisionsinarather indi-rect way. Rather than relying ontheories or beinglinkable to specificmodelsofthehuman-naturesystem,whichwouldallowto question,quantifyandorganisethecausalitieswithinthissystem, SDI-Cscanprovideaconceptualframework,openforinterpretation andapplication, thathelpstoorganisediagnosticand prescrip-tiveinquiry,andcanprovideaformof‘meta-theoreticallanguage’ enablingacomparisonbetweentheories(Ostrom,1999).Such con-ceptualframeworkindicators pursuetheobjective tostimulate informationflows that eventuallywilllead tocollective under-standingofemerging sustainabilitypatterns(Sonntag,2010).In thissense,itappearsadequatethatoursetofempiricallyexplored SDI-Csaretheory-drivenandrelyonastrongconceptuallanguage, allowingdiscussiononemergingparadigmsratherthanproviding unambiguousandtargetedpolicyadvicewithregardtocausalities. Theframeworkindicatorsappeartoaffectdecision-makers’ prob-lemdefinitions,andprovidenewperspectiveson–andinsightsinto –theproblemarea,insteadofprovidinginformationforaspecific momentofdecision, ortoahypotheticalsingledecision-maker. SDI-CsatEU-levelproduceprimarilyintangibleeffects(Gahinetal., 2003),fuellingdebatesandhelpingtoenlightenpolicyactorson sustainability-relatedissues.SDI-Csarenotpolicytoolsinthesense

7_The_different_indicator_sets_are_apparent_via_the_EU_Sustainable_Development

Strategy,theEU2020Strategy,theLisbonStrategy,andthesetofindicators pub-lishedandproﬁledbyDGEnvironment.

thattheywouldconstitutedirectlyapplicableevidencefor policy-making,butcaninsteadbeconsideredasconceptualreferences thatinformbroaderpolicyprocesses(Illner,1984).

In these realms, user factors are of a particular importance inunderstandingandinallowingtheoccurrenceofthis concep-tualinﬂuenceofcompositesustainabilityindicators.Asframework indicators,i.e.evokingparticularparadigmsonSD,SDI-Csneedto enablepolicy-makerstoquestionstabilisedcodesandrepertoires

and opennewwaysof representingrealitytothem.Our

quali-tativeapproachrevealedthattheacceptanceofsuchframework indicatorsbypolicy-makersisprimarilyconditionedbythelevel ofresonanceoftheindicatoritselfwiththerepertoiresofitsusers. TheimageofSDI-Csbeingammunitiontothelegitimisationof pos-itionsinon-goingpolicydebatesappearsthustobeparticularly supportedhere.

5.2. Indicatorfactors:initialisationofpolicydebatesvia discussingthequalityofSDI-Cs

Contrarytothehighdegreeofattentionwhichistraditionally givenbythecommunityofindicatorproducerstothe consolida-tionofthetechnicalandmethodologicalrobustnessofindicators, ourqualitativeexplorationoftheinﬂuenceofSDI-Csrevealsthat ‘indicatorfactors’matterlittleasdirectexplanatoryfactorforthe observeduseandinﬂuenceofindicators.Infact,anumberofSDI-Cs whichareacclaimedfortheirmethodologicalrobustnessremain

widelyunused bypolicy-makers,while others, whichmight be

ofpoorerscientificquality,havebeenadoptedtoacertainstance bypolicy-makersatEU-level.Noclearschemeofinfluencearises whentheinfluenceofSDI-Csisexploredagainsttheacknowledged (bythepolicy-makers)robustnessofthesetofSDI-Cs.However, the empirical, qualitative exploration shows consistency in the explanationofSDI-Cinfluencewhenplottedagainstthe acknowl-edgedmethodologicalconflictsandinterrogationswhichareraised

against speciﬁcSDI-Cs. In other words,the more

methodologi-calconflictsrevolvearoundanSDI-C,thebetterindicatorfactors explaintheobservedinfluence.Consequentially,theinfluenceof

an SDI-C seems to be conditioned more by the extend of the

negotiations betweeninterestgroups – aprocess duringwhich

methodologicalargumentationwouldbemobilised–thanbythe

effectivelevelofmethodologicalrobustnessoftheSDI-C(Cashetal., 2002,2003).

Morespeciﬁcally,theinterviewedEU-policymakerscriticised quiteextensivelythe‘EcologicalFootprint’(EF),consistently refer-ringtomethodologicalproblems,arguingforitsinadequacyduring

decision-making because EF is not pointing at speciﬁc policy

options,andregrettingthenon-inclusionintheEFofmany cru-cialaspectsofenvironmentalproblems(presumably,forexample, issueslinkedtotoxicity).Such‘technical’argumentsagainsttheEF havebeenraisedrepetitivelyinliterature,andaretodayhardlya matterofcontradiction.Interestingly,however,some interview-eesexplicitlyhighlightedthefactthatsomepolicyactorswould usethesemethodologicalissuesonEFtogroundtheiropposition toamoreformaladoptionoftheEcologicalFootprintbythe EU-institutions, whereas theirprimary reason ofopposition would actuallystemfromthefactthattheadoptionoftheEFwould con-siderablyincreasepressurefor aradicalchangeofpolicies–an objectiveforwhichEuropewouldintheirviewnotbeready.

Inparallel,aconsistentpartofourintervieweesseesintheHDI a“GDPhybrid,aGDPabitimproved,aGDPwithalesser empha-sisoneconomicvariables8_”._In_that_perspective,_HDI_might_still_be

viewedasaﬁrststeptomoveawayfromthetyrannyofGDP,but

(8)

L.Sébastien,T.Bauler/EcologicalIndicators35 (2013) 3–12 9 HDIisalsoaccusedofsomeformofhypocrisy,onlysimulatingthe

integrationofsocialaspectsintotheindexwhilekeepingGDPin averycentralposition.Intervieweesacknowledgedthatthe ori-ginofthefairlygoodimageofwhichHDIproﬁtsmightactuallybe linkedtothefactthatpeoplearegenerallyunawareofthepresence andcausalinﬂuenceofGDPwithinthecomposite’scalculus, lead-ingsomeofthepolicyactorstolabelHDIa“masquerade”.Hence, heretheactualmethodologicalcontentoftheSDI-Cappearstobe ofminorimportancethantheimageofitscontent.

Inbothexamples,themethodologicalconstructionofSDI-Cs– i.e.indicatorfactors–doesthusplayaroleintheexplanationof indicatorinfluence;butquiteindirectly.Whileindicatorfactorsdo notseemtoexertapositiveornegativeinfluenceonthechoiceof theindicatorstobeused,theyactuallyinitiatedebateaboutthe configurationofthevisionsofsustainabilitytheSDI-Csaimto rep-resent.Theminordirectexplanatoryforceofindicatorfactorscan beinterpretedaccordingtoPielke(2007)alongtwodimensions whichguidetheuptakeofscientificandexpertknowledgein poli-cymakingprocesses:1◦thedegreeofscientificuncertainty;and,2◦ thepotentialforvalueconsensus.Typically,SDI-Cscoreequivocally onbothcharacteristics:1◦theycontainahighdegreeofscientific uncertainty,forinstanceaboutthecorrectnessofcausalitieswhich materialiseintheaggregateindicators;2◦theyrevealarelatively highdegreeofconflictpotential,sinceeachSDI-Ccarriesitsspecific ‘worldvision’.Oncontroversialpolicyissues,withdisagreementon valuesandgoals,indicatorsthatenabletonarrowdowntherange ofpolicyoptionsshouldthushavemorepotentialforinfluenceon policyactorsthanSDI-C,which tendtoexacerbate theconflicts becausetheyallowtosupportdisputeontherepresentationtobe giventothegenericpolicyidea.Inotherwords,theEU-SDSarena isinneedforindicatorswhichhelptobreakdownthepolicyidea ofSDintopolicyoptions,butquiteonthecontrary,SDI-Ctendto openupthedebatebecausetheinter-institutionalinterpretationto begiventosustainabledevelopmentisnotconsensualyetinitself. TheacknowledgedpolicyinfluenceofSDI-Ccouldthusstemfrom thefactthattheirinherentindicatorcharacteristicstendto open-uppolicyvisions,ratherthanenabletoclose-downthestruggleof interpretations.

5.3. Policyfactors:howthenatureofthepolicyarenaofSD explainsfortheinﬂuenceofSDI-Cs

Theempiricalexplorationshowsthat‘policyfactors’arecentral inexplainingtheuseandinﬂuence–ortheirabsence–of indi-cators.In effect,themeagre reportedinﬂuenceof SDI-Cs could besimplyattributedtothenatureofthepolicydomain,seeing

in the use/inﬂuence dynamics of SDI-Cs a mirror-effect of the

dynamicsmorewidelyimposedonthepolicyagendaitself. ‘Sus-tainabledevelopment’asapolicyagendaisclaimedtohaveatleast partlyfailedtobecomeanoverarchingreferencepointinEU-level policymaking.Forinstance,unlikeotherEU-levelpolicyagenda, ‘sustainabledevelopment’isnotcentredonanarrowsetofclearly definedpolicyissues,butcoversarelativelybroadrangeoftopics ineconomic,socialandenvironmentalfields.Therelatively eva-sive,undefinednatureofthepolicy-agendaitselfwouldnoteasily

beconducivetoprovokeanintenseﬂowofuseandinﬂuenceof

SDI-Cs.

TheoveralldebateonSDI-Cs,ormorewidelyonalternatives toGDP,isproﬁtingfromaconsiderableandhigh-levelechoingin manyEU-institutions.Ontheground,itremainshoweverdifﬁcult toclarifywithpolicy-makerstheexactnatureofthe‘demandside’ forSDI-Cs.

Ontheonehand,thereisarelativelyweakstatedneedforSDI-C attheleveloftheinterviewedpolicy-makers.SDI-Cshavegained theirappreciation not tohelp policy-makers in designing poli-ciesnorguidingspeciﬁcsysteminterventions,butrathertogain

knowledgeofemergingoron-goingwiderpolicydebates(notably

onthenecessitytoreplaceorcomplementthehegemonicGDP).

Insinglecases,SDI-Cswereclaimedtobeusedmoreclassically

tocomparetheperformancesofregionsormember-countrieson

particularissues.FollowingIllner(1984),therelativeconstructed complexityofcompositeindicators–asopposedtotheirrelative

communicative simplicity– may suggest that SDI-Chave most

relevanceduringtheearlydescriptivephasesofapolicymaking process,i.e.toidentifygrosspolicyproblems.Theevidentfactthat thereisnoplethoraofnewpolicyproblems,henceofpolicydebates, thatemergeseveryyearontheEU-levelcouldaccountforthefact thatpolicy-makersexpressaweakdemandforSDI-Cs.

Ontheotherhand,policy-makersstatetobesubmergedbythe

sheernumberofSDI-Csbeingproposedfrommanyauthorsand

institutions.In thiscontext, handlingSDI-Cscanbeparticularly demandingfor policy-makers,becauseworkingwithSDI-Cs, i.e. usingthem,asksthematleastimplicitlytorallyaparticular strate-gicpositioning.Inthiscontext,SDI-CsatEU-levelareclaimedto functionastoolstojustifyaposterioriandsupportotherwise hard-enedpolicychoices,andnotasincentivebasetochangepoliciesor decideonspeciﬁcpolicyoptions.

From our explorations, the ‘demand’ for SDI-C by EU-level

policy-makerscannotbeunambiguouslyqualiﬁed,andseems

com-paratively sparse, undeﬁned, unsure and disparate against the

overwhelming presenceoftheSDI-C‘offer’.Anexplanatory fac-torforthedisequilibriumbetween‘supply’and‘demand’ofSDI-C canbefoundat theleveloftheoriginal motivationtoproduce andproposeSDI-Csinquestion.Anumberofthecurrently preva-lent,‘traditional’compositeindicators(amongwhichnottheleast GDP)wereelaboratedandproducedbyinstitutionallyappointed

experts upon speciﬁc demandby policy-makers facing speciﬁc

policy situations. Thesetraditional compositeindicators canbe acknowledged as ‘top-down’indicators, built onclear terms of references(co-)deﬁnedbytheirpre-identiﬁed‘users’.Whilesuch

demand-deﬁnedcompositeindicatorscontinuetobeelaborated

occasionally9_,_the_largest_part_of_the_offer_of_SDI-Cs_consists_of

indi-cators,whichareelaboratedforpolicy-makers,butnotby,norwith them.Theevolutionofthemoretraditional‘top-down’-movement

in the deﬁnitionand conﬁgurationof composite indicatorshas

indeed evolved,partlyundertheinﬂuenceof theemergenceof

sustainability-rootedprinciplessuchastheacknowledgementof

the extension of the knowledge-base beyond the one existing

withinexpertcircles(Fraseretal.,2006).Probablymoredirectly, theevolutionof ourinformatics toolsand thegeneralisation of digitalaccesstoabroadsetofdatatremendouslyfacilitatedthe capacity ofnon-institutionalactors(e.g.NGOs)toengageinthe elaboration ofcompositeindicators.Asaresult,composite indi-catorsareincreasingly proposedtopolicy-makersbyagrowing varietyofnon-institutionalpolicyactors10_._An_important_number

of currentSDI-Csaretherefore neithertheresultof‘top-down’, expert-driveninstitutionalexercises,noraretheseindicators rely-ingonparticipatoryprocessesandcouldbeshelvedas‘bottom-up’, citizen-deﬁned,localandspeciﬁcindicators(Fig.2).

Inﬁne,agrowingnumberofSDI-Cscouldbelabelled ‘middle-up’ and ‘middle-down’indicators (Sebastienand Bauler,2010); theyareinitiatedbystakeholderorganisationssuchasNGOsor ThinkTanks,occasionallytheyoriginateinindependent,academic exercisesbutwhicharethenfurtherdeveloped,ﬂeshedand com-municatedbystakeholderorganisations.These‘middle’actorsare

9_For_a_recent_example,_see_for_instance_the_on-going_construction_of_{material-ﬂow}

indicators–andtheunderlyingmaterial-ﬂowaccountingframeworks–atEU-level.

10_See_for_instance,_the_Happy_Planet_Index_by_the_New_Economics_Foundation_,_or

theIndexforSustainableEconomicWelfarewhichwasoriginallysupportedby RedeﬁningProgress.

(9)

Fig.2. DiffusionofSDI-Cby“middle-actors”totopandbottomactors.

largely independent, non-institutionalised organisations, which functionasintermediateactorsbetweencivilsocietyatlargeand thepolitical/institutionalpolicyspheres.Inthissense,the ‘Ecolog-icalFootprint’isagoodexample:elaboratedandconceptualised duringaPhDthesis,EFwasrapidlytakenupbytheWorldWildlife Fundandistodayverycontroversiallydiscussed(andtakenupor rejected)instatisticalofﬁces.Inawiderinterpretationofpolicy processes,the‘Mittelfeld’actors(i.e.NGOs,ThinkTanks,engaged scientists)aretheoneswhopre-conﬁgurethevarious‘visions’for society (e.g.of thelimits of ourenvironmental resources), and whoreachouttoadvocatetheirvisionstowards‘top’(i.e.policy actors)and ‘bottom’(i.e.societyat large).In thisprocess,

com-posite indicators are much-welcomed tools which supportthe

communicationofworldvisions.AccordingtoWeiss(1999),the independenceofthe(information)media,aswellasthestrength andthestatusoftheNGOsandthecivilsocietyingeneraltendto promotecriticaldiscussioninsociety,therebypotentially increas-ingtheinfluence ofindicators.More particularly,in thecaseof oursetofSDI-Cs, mostof the‘Mittelfeld’-actors atthebasis of theSDI-Cs have first diffused theirideas(and indicators) hori-zontally(i.e.betweeneachother)inordertobuildanetworkof coalitionalactors,withtheaimtoproposeatalater stagetheir indicatorstopolicymakers (‘middle-up’)and/or tocivilsociety (‘middle-down’).Along this readingof ‘Mittelfeld’policyactors coalitioningbehindparticularindicators,thedegreeoflegitimacy andcredibilityoftheindicator-proposingactorsplaysamajorrole inexplainingthepotentialofinfluenceoftheindicators(Boulanger, 2007).

Insuchacontextofconﬂictingandevolvingpolicyarenas, strug-glingforattentionwiththesupportofindicators,theinterviewed policy-makersatEU-levelclearlystatedtheircuriosityinwellbeing

asanewoverarchingpolicyconcept,andtheirdisinterestinthe somewhatoverused conceptofsustainabledevelopment.The so-calledStiglitzreport(Stiglitzetal.,2010)helpedtoinitiatethis recentreflectiononthewaysocietiesshouldthinktheirfutures inlinewithrethinkingtheirindicators.Thenotionsofqualityof life,wellbeing,andsocialprogress–andtheassociatedindicators– areslowlytakingtheroleofsustainabledevelopmentasthe over-archingnormativeconceptofthe‘post-GDPsociety’.Likethe1987 BrundtlandreportforSD,theStiglitz-reportinitiatedanemerging reflectiononthewayEuropeshouldthinkitsfuture,manage pri-oritiesandmeasuredevelopment.Wellbeingandsocialprogressare putinthefirstrow,whereasthecoreoftheSD-conceptualisation, i.e.environmentalandresource-relatedobjectives,arerelegated

into means for wellbeing. This (r)evolution in policy arenas

(Boulanger, 2007) – in turn – appears to raise the interest of ourpolicy-makersintheforthcomingwellbeing-relatedcomposite indicators;andcouldexplainforsomeofthestateddisinterestin SDI-Cs.

6. Conclusion

Sustainabilityindicatorsareseldomuseddirectlyand

instru-mentally,butfarmoreofteninﬂuencedecision-makingthrough

variousindirectandoftenslowprocesses.Amongthemostvisible examplesofindirectinﬂuenceweresituationsinwhichindicators wereusedasammunitioninpoliticaldebates.Experts(whether

independentorgovernmental)oftenseemtoproduceindicators

withoutaclearintendeduseoruserinmind,andwithoutregard tothepotentialinﬂuence of theindicators.Combinedwiththe abundanceofindicators,thisincreasestheriskthatpolicymakers useonlythoseindicatorsthatsuittheirowninterests,while con-cealingother,potentiallymorerelevantones.Mismatchbetween supplyanddemandishowevernotsurprising.TheSDI-Cscanbe described as tools speciﬁcallydesigned by groups of politically engagedexpertsforpurposesofpoliticalpersuasion.

While most of the academic attention and political debate

aroundindicatorshastendedtofocuson‘indicatorfactors’,such quality attributes actually matteredrelatively littleas

determi-nantsofindicatorinﬂuence.Sometimesconﬂictsbetweenworld

visionswerehiddenbehindmethodologicaldebatesonindicators. Andyet,eventhough‘indicatorfactors’werethusnotdecisivein determiningindicatorinfluence,keyusergroupssawhightechnical andscientificqualityasanessentialprerequisiteforthe determi-nationof theirintended useofindicators.Recommendationsto ensurethatdemandandsupplymeeteachotherincludethe sug-gestionthatindicatorproducersshouldnotdevotealltheirenergy tomethodologicalaspectsbutstudythepotentialreceptivityof theirindicator.Theyshouldalsotrytometaphorically‘optimise thesignal’,‘improvethereception’inthepolicysystemor‘increase thebandwidth’(DaCostaet al.,2008) toincrease collaboration betweenproducersandusers.ButasRice(2003)recalls,inanoisy environmentliketheoneofSD,aperfectsignalmightbeimpossible toget.Chateauraynaud(2011)arguesthatthequalityofevidenceis definedbythenumberofcontroversiesthatitgoesthroughduring itslifetime.Thisviewrejectstheideathatthequalityand robust-nessofevidencewouldlieexclusivelyinitstechnicalqualityandin theindependenceofitsproducer,andinsteadcallsattentiontothe processesofevidence-construction.Fromthisperspective, strate-gicandpoliticaluseofindicators,manipulationorevenabuseof indicatorsisnotnecessarilyaproblem,butratheranessentialpart oftheproductionofvalidandreliableevidence.

Likewise,inexplainingtheuseandinﬂuenceofSDI-Cs,‘user factors’(beliefsandrepresentationsofpolicyactors)and‘policy fac-tors’(institutionalcontext)werecrucial,whereas‘indicatorfactors’ (qualityoftheindicators)playedonlyalimitedrole.Ourﬁndings calltogivegreaterattentiontothevariousactorsinvolvedaswell astothegeneralpoliticalandinstitutionalcontextsinwhich indi-catorsarebeingproduced.

OurresultsmirrorthefactthatSDisnotaunifyingpolicyidea attheEU-level.Ifthereseemstobeaconsensusaboutthe the-oreticalideaofSD,ouranalysesrevealthatthewaystoachieve SDarestillcontroversialamongstproducersofindicators, stake-holdersandevenamongstEUinstitutions.AsSDisnot(anymore?) theoverarchingpolicyframeworkatEU-level,SDI-Csarerarelyput forward;thecurrentreferencepolicydocumentsappeartobethe LisbonStrategyandtheEU2020strategyratherthantheEU-SDS. ThelittlereferencegiventoSDisacknowledgedtobefurtherfading out,andreplacedlittlebylittlebyanewerframeworkofreference, i.e.thepolicyideaof‘wellbeing’.Atindicatorlevel,thereferences to‘wellbeing’havebeenadvocatedandpopularisedprominently bytheStiglitz-Sen-Fitoussireport.

SDI-Csarecontroversialbecauseofthevariousvisionsof

sus-tainability existing in the policy arenas. We show that SDI-C

are ammunitionat theservice of variousstakeholders to legit-imisea particularpoliticalandsocietalagenda,i.e.their‘vision’

(10)

L.Sébastien,T.Bauler/EcologicalIndicators35 (2013) 3–12 11 ofsustainability.SDI-Care‘frameworkindicators’,enhancingthe

conceptualandsymbolicinﬂuenceofdata.Theyaretheory-driven

andhelpdiscussingnewparadigmswithoutnecessarily

suppor-tingsystematicpracticalpolicyoutputs.Theobjectiveistoreach sharedunderstandingandpolicylearning.Onepositiveeffectis thatSDI-Cenhancediscussionsonthedesirablestateofour soci-etyandenvironment.Ontheotherhand,thesecontroversieslimit theuseofSDI-Cbecauseconsensusisdifﬁculttoachieveregarding suchindicators.Theseoverallopposingviewpointsgeneratea rela-tivelyvagueanduncleardemandfrompolicymakersforcomposite

indicators. On the one hand,high-level policy makers seem to

appreciatethefactthatSDI-Careratherinadequatewhenitcomes todesignpolicies,butarerelativelysuitedtocapturerapidlythe stakesof anissue.On the otherhand,policymakers are being overwhelmedbyindicatorscomingfromallsidesandpresenting variousfacetsofapolicyproblem.

Thereremainmajorgapsbetweenindicatorcreatorsandusers,

leading to misunderstandings between actors. However, there

were also internal quarrels within EU institutions about

sus-tainabilityindicators.ThedifferentinstitutionsatEU-leveldonot

have the same visions of what should be measured regarding

sustainabilityinEurope.AscontroversyremainswithinEU’s insti-tutions,manydiscussionsarebeingruninternallyandSDI-Csrather exacerbateexistingpolicyconﬂictsbetweenEU-institutions. Nev-ertheless,newtypesofgovernanceprocessesaretakingplacein paralleloftheEU-debate,ledbywhatwecall“middleactors”. SDI-Carecurrentlybeingproposedactivelybythesetypesofactors(i.e. ‘Intermediate’-actorssuchasNGOs,thinktanks,scientists,etc.)to decisionmakersaswellastocitizens.Werefertothese percola-tionprocessesasbeing‘middle-up’and‘middle-down’indicators. Thislatterobservationshouldbegivenfurtherattentionbyfuture explorations:indicatorpathwaysthroughsocialnetworksandthe longitudinalstudyofthepercolationfromonepolicyactor-levelto anothermightwellprovideforafurtherimportantreﬁnementof thepolicyuseofcompositeindicators.

Acknowledgments

Thepresentpaperisasynthesisofsomeoftheresultsacquired duringthePOINT(PolicyinﬂuenceofIndicators)projectﬁnanced

undertheEuropeanCommission’s7thFrameworkprogram(Grant

Agreementnumber217207).Moreinformationandmaterialcan

beobtainedat:www.POINT-EUFP7.info.Wewouldliketothank

the2anonymousreviewersfortheirconstructivefeedbackonan earlierdraft.

References

Amara,N.,Ouimet,M.,Landry,R.,2004. Newevidenceoninstrumental, concep-tual,andsymbolicutilizationofuniversityresearchingovernmentagencies. Sci.Commun.26(1(September)),75–106.

Ayres,R.U.,2000. Commentaryontheutilityoftheecologicalfootprintconcept. Ecol.Econ.32,347–349.

Bauler,T.,2012.Ananalyticalframeworktodiscusstheusabilityof(environmental) indicatorsforpolicy.Ecol.Indic.17,38–45.

Boulanger,P.-M.,2007.Politicalusesofsocialindicators:overviewandapplication tosustainabledevelopmentindicators.Int.J.Sust.Dev.10(1–2),14–32.

Cash,D.,Clark,W.,Alcock,F.,Dickson,N.,Eckley,N.,Jäger,J.,2002.Salience, Cred-ibility,LegitimacyandBoundaries:Linkingresearch,assessmentanddecision making.In:JohnF.KennedySchoolofGovernment,HarvardUniversity,Faculty ResearchWorkingPapersSeries(November2002(RWP02-046)).

Cash,D.W.,Clark,W.C.,Alcock,F.,Dickson,N.M.,Eckley,N.,Guston,D.H.,Jager,J., Mitchell,R.B.,2003.Knowledgesystemsforsustainabledevelopment.Proc.Natl. Acad.Sci.100,8086–8091.

CEC-COM,2009. GDPandbeyond;measuringprogressinachangingworld. Com-municationfromtheCommissiontotheCouncilandtheEuropeanparliament, 433ﬁnal.

Chateauraynaud,F.,2011.Argumenterdansunchampdeforces.Essaidebalistique sociologique,Pétra,Paris.

Devitt,J.,DeFusco,D.,2002.FinlandRanksHighestinEnvironmentalIndex;U.S. Lags. In: PressRelease of the EnvironmentalSustainability Indexproject.

http://www.ciesin.columbia.edu/indicators/ESI/pressrel.html

FAIRmanifesto,2008.Froma“somehavemuch”societytoasustained “well-being-for-all”society.FAIR(ForumforotherIndicatorsofwealth)(coordinatedby HeleneCombe).

Fraser,E.,Dougill,A.J.,Mabee,W.E.,Reed,M.,McAlpine,P.,2006.Bottomupandtop down:analysisofparticipatoryprocessesforsustainabilityindicator identiﬁca-tionasapathwaytocommunityempowermentandsustainableenvironmental management.J.Environ.Manag.78(2),114–127.

Flyvbjerg,B.,1998. RationalityandPower:DemocracyinPractice.Universityof ChicagoPress,Chicago,IL.

Gahin,R.,Veleva,V.,Hart,M.,2003.Doindicatorshelpcreatesustainable commu-nities?LocalEnviron.8(6),661–666.

Gallopin,G.C.,1996. Environmentalandsustainabilityindicatorsandthe con-ceptofsituationalindicators.Asystemsapproach.Environ.Model.Assess.1, 101–117.

Gudmundsson,H.,2003.Thepolicyuseofenvironmentalindicators–learningfrom evaluationresearch.J.TransdisciplinaryEnviron.Stud.2(2),1–12.

Hardi,P.,Zdan,T.,1997.AssessingSustainableDevelopment:PrinciplesinPractice. InternationalInstituteforSustainableDevelopment,Winnipeg.

Mark, M.M.,Henry,G.T.,2003. Beyond use:understandingevaluation’s inﬂu-ence on attitudes and actions. American Journal of Evaluation 24 (3), 293–314.

Hezri,A.A.,Dovers,S.R.,2006. Sustainabilityindicators,policy,governance:issues forecologicaleconomics.Ecol.Econ.60,86–99.

Holden,M.,2008. Sociallearninginplanning:seattle’ssustainable.Development codebooks.Progr.Plan.69(2008),1–40.

Illner,M.,1984.Onfunctionaltypesofindicatorsinsocialplanning.Soc.Indic.Res. 14(3),275–285.

Innes,J.E.,Booher,D.E.,2000. Indicatorsforsustainablecommunities:astrategy buildingoncomplexitytheoryanddistributedintelligence.Plan.TheoryPract. 1(2),173–186.

Innes,J.E.,1998.Informationincommunicativeplanning.J.Am.Plan.Assoc.64(1), 52–63.

JointResearchCenter(JRC),2010.IndicatorsforLisbonpost-2010.Businessasusual? JRCScientiﬁcandtechnicalreports(EUR57104).

Kingdon,J.W.,1984.Agendas,AlternativesandPublicPolicies.LittleBrown,Boston.

Lehtonen,M.,2004.Theenvironmental-socialinterfaceofsustainabledevelopment: capabilities,socialcapital,institutions.Ecol.Econ.49,199–214.

Lehtonen, M., 2005. OECD Environmental performance review programme: accountability(f)orlearning?Evaluation11(2),169–188.

Leknes,E.,2001. TherolesofEIAinthedecision-makingprocess.Environmental ImpactAssessmentReview21(4),309–334.

McCool,S.F., Stankey,G.H., 2004. Indicators ofsustainability:challenges and opportunities at the interfaceof science and policy. Environ. Manag. 33, 294–305.

Ortega-Cerdà,M.,2005.Sustainabilityindicatorsasdiscursiveelements.In:Paper presentedatthe6thInternationalConferenceoftheEuropeanSocietyfor EcologicalEconomics,Lisbonhttp://ecoman.dcea.fct.unl.pt/projects/esee2005

(accessed13.03.09).

Ostrom,E.,1999.Institutionalrationalchoice:anassessmentoftheIADframework. In:Paul,S.(Ed.),TheoriesofthePolicyProcess.WestviewPress1997,Boulder, CO.

OrganisationforEconomicCooperationDevelopment(OECD),2011. How’slife? MeasuringWellbeing.OECD,Paris.

Owens,S.,Rayner,T.,Bina,O.,2004.Newagendasforappraisal:reﬂectionsontheory, practice,andresearch.Environ.Plan.A36(11),1943–1959.

Parris,T.,Kates,R.,2003. Characterizingandmeasuringsustainabledevelopment. Ann.Rev.Environ.Res.28(13),1–28.

Patton,M.,1997.Utilization-FocusedEvaluation:TheNewCenturyText.Sage, Thou-sandOaks,CA.

Pielke,R.A.,2007.TheHonestBroker:MakingSenseofScienceinPolicyandPolitics. CambridgeUniversityPress.

Pollitt,C.,2006. Performanceinformationfordemocracy.Miss.Link?Eval.12(1), 39–56.

Pregernig,M.,2000.Puttingscienceintopractice:thediffusionofscientiﬁc knowl-edgeexempliﬁedbytheAustrian‘ResearchInitiativeAgainstForestDecline’. For.PolicyEcon.1(2),165–176.

Rice,J.,2003. Environmentalhealthindicators.OceanCoastalManagement46, 235–259.

Romsdahl,R.J.,2005.PoliticalDeliberationandE-ParticipationinPolicy-Making. CLCWeb:ComparativeLiteratureandCulture7.2.http://docs.lib.purdue.edu/ clcweb/vol7/iss2/7

Rosenström,U.,2006.Exploringthepolicyuseofsustainabledevelopment indica-tors:interviewswithFinnishpoliticians.J.TransdisciplinaryEnviron.Stud.5 (1–2)http://www.journal-tes.dk

Rosenström,U.,Lyytimäki,J.,2006. Theroleofindicatorsinimprovingtimeliness ofinternationalenvironmentalreports.Eur.Environ.16,32–44.

Rydin,Y.,2007. Re-examiningtheroleofknowledgewithinplanningtheory.Plan. Theory2007,52–68.

Sabatier, P., 1991. Political science and public policy. Polit. Sci. Polit. 24, 144–146.

Sager,T.,Ravlum,I.A.,2005.Fromprojectstostrategies:atransactioncostapproach topoliticians’problemswithstrategictransportplanning.Plan.TheoryPract.6 (2(June)),213–232.

(11)

Sebastien,L.,Bauler,T.,2010. Istherearealdemandforcompositeindicatorsfor sustainability?In:ProceedingsoftheInternationalConferenceofEcological Economics,Oldenburg,August22–26.

Sonntag,V.,2010. DesigningSustainabilityIndicatorFrameworksforInformation Flow:ACaseStudyofB-Sustainable.In:AppliedResearchQualityLife,The Inter-nationalSocietyforQuality-of-LifeStudies(ISQOLS)2010.

Stiglitz,J.E.,Sen,A.,Fitoussi,J.-P.,2010. Mismeasuringourlives.WhyGDPdoesn’t addup.In:TheReportbytheCommissionontheMeasurementofEconomic PerformanceandSocialProgress.NewPress,NewYork/London.

Turnhout,E.,Hisschemöller,M.,Eijsackers,H.,2007.Ecologicalindicators:between thetwoﬁresofscienceandpolicy.EcologicalIndicators7,215–228.

Valovirta,V.,2002.Evaluationutilizationasargumentation.Evaluation8(1),60–80.

VanderKnaap,P.,2006. Responsiveevaluationandperformancemanagement. Overcomingthedownsidesofpolicyobjectivesandperformanceindicators. Evaluation12(3),278–293.

VanderMeer,F.-B.,1999. Evaluationandthesocialconstructionofimpacts. Eval-uation5,387–406.

Vedung,E.,1997. PublicPolicyandProgramEvaluation.TransactionPublishers, NewBrunswick(NJ),USA.

Weiss,C.,Murphy-Graham,E.,Birkeland,S.,2005. Analternateroutetopolicy inﬂuence:howevaluationsaffectD.A.R.E.Am.J.Eval.26(1),12–30.

Weiss,C.,1999. Theinterfacebetweenevaluationandpublicpolicy.Evaluation5 (4),468–486.