Guidelines on data disaggregation for SDG Indicators using

Preparing to load PDF file. please wait...

0 of 0
100%
Guidelines on data disaggregation for SDG Indicators using

Transcript Of Guidelines on data disaggregation for SDG Indicators using

Guidelines on data disaggregation for SDG Indicators using survey data
1

2

Guidelines on data disaggregation for SDG Indicators using survey data
Food and Agriculture Organization of the United Nations Rome, 2021 iii

Required citation: FAO. 2021. Guidelines on data disaggregation for SDG Indicators using survey data. Rome. https://doi.org/10.4060/cb3253en
The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific co mpanies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.
The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO.
ISBN 978-92-5-133942-8 © FAO, 2021
Some rights reserved. This work is made available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BYNC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo/legalcode).
Under the terms of this licence, this work may be copied, redistributed and adapted for non-commercial purposes, provided that the work is appropriately cited. In any use of this work, there should be no suggestion that FAO endorses any specific organization, products or services. The use of the FAO logo is not permitted. If the work is adapted, then it must be licensed under the same or equivalent Creative Commons licence. If a translation of this work is created, it must include the following disclaimer along with the required citation: “This translation was not created by the Food and Agriculture Organization of the United Nations (FAO). FAO is not responsible for the content or accuracy of this tra nslation. The original [Language] edition shall be the authoritative edition.”
Disputes arising under the licence that cannot be settled amicably will be resolved by mediation and arbitration as described in Article 8 of the licence except as otherwise provided herein. The applicable mediation rules will be the mediation rules of the World Intellectual Property Organization http://www.wipo.int/amc/en/mediation/rules and any arbitration will be conducted in accordance with the Arbitration Rules of the United Nations Commission on International Trade Law (UNCITRAL).
Third-party materials. Users wishing to reuse material from this work that is attributed to a third party, such as tables, figures or images, are responsible for determining whether permission is needed for that reuse and for obtaining permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.
Sales, rights and licensing. FAO information products are available on the FAO website (www.fao.org/publications) and can be purchased through [email protected] Requests for commercial use should be submitted via: www.fao.org/contact-us/licence-request. Queries regarding rights and licensing should be submitted to: [email protected]
iv

Contents
Foreword ................................................................................................................................................ix Acknowledgements ................................................................................................................................ x Chapter 1. Introduction .......................................................................................................................... 1 1.1. Background ...................................................................................................................................... 1 1.2. Scope of these Guidelines................................................................................................................. 3 1.2.1. The main focus of these Guidelines ................................................................................................ 3 1.2.2. Some useful concepts..................................................................................................................... 5 1.2.3. Summary of the Guidelines ............................................................................................................ 5 1.2.4. Focusing on a specific application .................................................................................................. 6 Chapter 2. A strategic plan for data disaggregation ................................................................................ 9 2.1. Why is a strategic plan for data disaggregation necessary? ............................................................... 9 2.2. The four pillars of the strategic plan.................................................................................................. 9 2.3. Relationships among the pillars ...................................................................................................... 10 2.4. Making the strategic plan effective ................................................................................................. 12 2.5. Focus on the strategic actions ........................................................................................................ 13 2.5.1. Harmonizing and standardizing the statistical processes .............................................................. 13 2.5.2. Specific actions on the statistical plan at the country level............................................................ 15 2.6. Chapter wrap-up and main recommendations ................................................................................ 16 Chapter 3. Direct sampling strategies for data disaggregation ............................................................. 17 3.1. Introduction ................................................................................................................................... 17 3.2. Basic theory on sampling and estimation ........................................................................................ 18 3.3. Direct estimation for the data disaggregation ................................................................................. 21 3.3.1. Repeated sampling ...................................................................................................................... 21 3.3.2. The model-based approach.......................................................................................................... 29 3.3.3. Extensions to parameters different from the totals....................................................................... 32 3.4. Traditional sampling techniques ..................................................................................................... 34 3.4.1. Oversampling............................................................................................................................... 34 3.4.2. Deeper stratification .................................................................................................................... 36 3.4.3. Multiphase sampling with a screening of respondents ................................................................. 38
iii

3.5. Marginal stratification designs ........................................................................................................ 39 3.5.1. Motivating example..................................................................................................................... 39 3.5.2. General overview ......................................................................................................................... 41 3.5.3. Balanced sampling for marginal stratification .............................................................................. 41 3.5.4. Marginal stratification design for two-stage or two-phase sampling designs................................ 43 3.6. Indirect sampling and multisource sampling designs ....................................................................... 45 3.6.1. General background..................................................................................................................... 45 3.6.2. Indirect sampling: basic methodology .......................................................................................... 48 3.6.3. Multisource sampling................................................................................................................... 50 3.7. Summary of the main recommendations ........................................................................................ 52 Appendix A3.1. ...................................................................................................................................... 53 Chapter 4. Computing the accuracy of disaggregated data................................................................... 54 4.1 Introduction .................................................................................................................................... 54 4.1.1. Why must sampling errors be estimated? .................................................................................... 54 4.1.2. The measure of accuracy.............................................................................................................. 54 4.1.3. Evaluating accuracy ..................................................................................................................... 55 4.2. Basic theory: the measures of accuracy .......................................................................................... 56 4.2.1. Sampling variance........................................................................................................................ 56 4.2.1.1. Sampling variance of the balanced sampling............................................................................. 58 4.2.2. Model variance ............................................................................................................................ 59 4.2.3. Global variance............................................................................................................................ 60 4.3. Case study: SDG Indicator 2.1.2 – Prevalence of moderate or severe food insecurity in the population, based on the FIES ............................................................................................................... 62 4.3.1. Brief description of the methodology for SDG Indicator 2.1.2........................................................ 62 4.3.2. Results by subpopulation: gender................................................................................................. 64 4.4. Summary of main recommendations .............................................................................................. 67 Appendix A4.1. Estimate of the global variance component 𝐸𝑃𝑉𝑀𝑌𝑑................................................... 68 Appendix A4.2. Indicator of the prevalence of food insecurity ............................................................... 69 Appendix A4.3. Estimates of confidence intervals for different countries............................................... 73 Chapter 5. Integrated use of two surveys ............................................................................................. 76 5.1. Introduction ................................................................................................................................... 76 5.2. Methodology .................................................................................................................................. 76 5.2.1. The projection estimator .............................................................................................................. 76 5.2.1.a. Bias and variance...................................................................................................................... 78
iv

5.2.1.b. Domain estimation.................................................................................................................... 79 5.2.1.c. Extensions ................................................................................................................................. 80 5.2.2. Selection of auxiliary variables ..................................................................................................... 82 5.2.3. Model assumptions and performance .......................................................................................... 82 5.3. Case study: food insecurity in Malawi ............................................................................................. 83 5.3.1 Background .................................................................................................................................. 83 5.3.2. Available auxiliary information .................................................................................................... 84 5.3.3. Projection model.......................................................................................................................... 85 5.3.4. Variable selection ........................................................................................................................ 87 5.3.5. Results ......................................................................................................................................... 88 5.3.4.1. Results of the prevalence of severe food insecurity .................................................................... 89 5.3.4.2. Results for prevalence of moderate or severe food insecurity..................................................... 93 5.4. Lessons learned .............................................................................................................................. 96 Appendix A5.1. List of auxiliary variables .............................................................................................. 97 Appendix A5.2. Results – R output for glm () function............................................................................ 99 A5.2.1. Severe food insecurity ................................................................................................................ 99 A5.2.2. Moderate or severe food insecurity............................................................................................ 99 Chapter 6. Small area estimation techniques ..................................................................................... 100 6.1. Introduction ................................................................................................................................. 100 6.2. Process flow for computing small area estimates.......................................................................... 101 6.2.1. Clarification for the identification and prioritization of needs ..................................................... 101 6.2.2. Calculation of direct estimates together with basic design smoothing techniques ...................... 103 6.2.3. Enhancement of the basic design smoothing techniques ............................................................ 103 6.3. Parameters of interest and the working model ............................................................................. 105 6.3.1. Notation .................................................................................................................................... 105 6.3.2. The General Working model....................................................................................................... 106 6.4. Construction of the vector of target variables and domains .......................................................... 109 6.4.1. Replaceable and non-replaceable variables................................................................................ 109 6.4.2. Partially replaceable variables ................................................................................................... 109 6.4.3. Unit-level auxiliary variables ...................................................................................................... 110 6.4.4. Unit-level auxiliary variables with error and proxy measurement ............................................... 110 6.4.5. Area-level auxiliary variables...................................................................................................... 111 6.5. A classification of estimators......................................................................................................... 111 6.5.1. How the estimator gains strength from other domains .............................................................. 112
v

6.5.2. How the estimator uses the data observed in the domain .......................................................... 113 6.5.3. The classification adopted.......................................................................................................... 114 6.6. Multivariate projection estimators................................................................................................ 116 6.6.1. Preamble ................................................................................................................................... 116 6.6.2. Alternative strategies for defining the auxiliary variables for direct and indirect estimators ....... 117 6.6.3. Projection estimator of 𝒚 +....................................................................................................... 117 6.6.4. Projection estimator of 𝒚 + ′ ..................................................................................................... 118 6.7. Summary of the main recommendations....................................................................................... 118 References .......................................................................................................................................... 119 Annex 1: R packages for data disaggregation ....................................................................................... 126
Figures
Figure 1.1 Availability of disaggregated data by SDG indicator………………………………………………………………..2
Figure 2.1. Sequence of the actions, with different scenarios………………………………………………………………..11
Figure 2.2. Factors that make the strategic plan effective..……………………………………………………..…………….12
Figure 3.1. Example of links between a frame of households and the target population of agricultural holdings in the household sector………………………………………………………………..…………………46
Figure 3.2. Example of multisource sampling: target population covered by the union of two sources…….………………………………………………………………………………………………………………………………...47
Figure 4.1. Margins of error for moderate or severe food insecurity prevalence, in male and female populations……………………………………………………………………………………………………………..64
Figure 4.2. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Afghanistan, Gambia, Eswatini, Angola and Togo, total and by gender, 2016–2018................................................................................................66
Figure 4.3. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Costa Rica, Kyrgyzstan, Uruguay, Moldova and Mongolia, total versus by gender, 2016–2018......................................................................................66
Figure A.4.1. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2), total and by gender, 2016–2018..............................................................73
Figure A.4.2. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in the Russian Federation and Iceland, total versus by gender, 2016–2018.....................................................................................................................73
vi

Figure A.4.3. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Afghanistan and Gambia, total and by gender, 2016–2018...................74 Figure A.4.4. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Kyrgyzstan and Uruguay, total and by gender, 2016–2018.................................................................................................................74 Figure A.4.5. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Moldova, Mongolia, Mexico, Tajikistan and Argentina, total and by gender, 2016–2018.........................................................................75 Figure A.4.6. Confidence interval for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2) in Eswatini, Egypt, Bangladesh, Uruguay, the Russian Federation and Iceland, total and by gender, 2016–2018........................................75 Figure 5.1. Projection estimator..................................................................................................................77 Figure 5.1a. Projection estimator for subsampling......................................................................................82 Figure 5.2. Histogram of the prevalence of moderate and severe food insecurity.....................................86 Figure 5.3. Level of importance of the auxiliary variables for severe food insecurity.................................90 Figure 5.4. Level of importance of the various levels of auxiliary variables for severe food insecurity.................................................................................................................................90 Figure 5.5. Level of importance of auxiliary variables for moderate food insecurity..................................93 Figure 5.6. Level of importance of various levels of auxiliary variables for moderate food insecurity............................................................................................................................94
Tables
Table 1.1. Data disaggregation dimensions and categories for SDG Indicator 2.1.2………………..............…….8 Table 3.1. Sample sizes n needed to guarantee the minimum threshold 𝑛𝑑∗ by percent values of the subpopulation proportion (𝑃𝑑) ……………..……………………………………………………………34 Table 3.2. Sample sizes n needed to guarantee the minimum threshold n_d^* by percentage values of the subpopulation proportion (P_d)…………………………….………………………………….36 Table 3.3. Example of stratification by region ……………………………………………………………………..………………..37
vii

Table 3.4. Example of marginal stratification design. Fixed sample of municipalities and individuals by region and living place………………………………………………….……………………39 Table 3.5. Example of marginal stratification design: selected municipalities and sample of individuals (in red brackets) in each cross-classification cell……………………………………..…….40 Table 4.1. Average margins of error for moderate or severe food insecurity prevalence, in male and female populations…………………………………………………………………………………….…..65 Table 4.2. Relative standard error for the prevalence of moderate or severe food insecurity (SDG Indicator 2.1.2), total and by gender, 2016–2018…….…………………………………………..65 Table 5.1. Variance-inflation factors for prevalence of severe food insecurity…..................................…..91 Table 5.2. Estimates for prevalence of severe food insecurity…...........................................................…..92 Table 5.3. Estimates for prevalence of moderate or severe food insecurity…..................................……...95 Table 6.1. Summary table of the projection estimators considered…………………………………………………….115
Boxes
Box 1.1. Disaggregation matrix for FAO-relevant SDG 2 and SDG 5 indicators………………………….………………4 Box 2.1. Examples of relationships among the pillars……………………………………………………………………………..10 Box 3.1. Two-stage stratified sampling design ………………………………………………………………………………..…….20 Box 3.2. Example of a stratified two-stage probability-proportional-to-size sampling without replacement……….…………………………………………………………………………………………………………………….36 Box 3.3. Example of a deeper stratified two-stage PPS sampling without replacement……………………..….37 Box 3.4. Examples of auxiliary variables for the balanced sampling illustrated in Table 3.4……………….…..42 Box 3.5. Examples of the concept of multiplicity……………………………………………………………………………….…..48 Box 3.6. Examples of indirect sampling for hard-to-reach populations…………………………………………….…….50 Box 4.1. Estimate of the variance for stratified two-stage sampling designs.............................................58
viii
PrevalenceFood InsecurityGenderVariablesFao