# Chapter10.IdentificationandDataAssessment.pptx

Identification and Data Assessment

Chapter 10

© 2019 McGraw-Hill Education. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or distribution without the prior written consent of McGraw-Hill Education

Learning Objectives

Explain what it means for a variable’s effect to be identified in a model

Explain extrapolation and interpolation and how each inherently suffers from an identification problem

Distinguish between functional form assumptions and enhanced data coverage as remedies for identification problems stemming from exploration and interpolation

Differentiate between endogeneity and types of multicollinearity as identification problems due to variable co-movement

Articulate remedies for identification problems and inference challenges due to variable co-movement

Solve for the direction of bias in cases of variable co-movement

‹#›

© 2019 McGraw-Hill Education.

The table below shows a subsample of rocking chair data

Your goal is to estimate the average treatment effect of price on sales. On average, when price increases by $1, what is the effect on sales of rocking chairs?

Assessing Data via Identification

‹#›

© 2019 McGraw-Hill Education.

A parameter (e.g., β) is identified within a given model if it can be estimated with any level of precision given a large enough sample from the population

Suppose we assume the data-generating process as:

Salesi = α + βPricei + Ui

Within this model, we are interested in accurately estimating β.

A parameter is identified if, for a given confidence level K ( < 100% ) and a given “length” L, we can build a confidence interval that contains β with length less than l and confidence level of K, given a large enough sample of data

Assessing Data via Identification

‹#›

© 2019 McGraw-Hill Education.

Identification Example

Define p as the probability of rolling a 3 on any single roll of a die.

Define X to be number of 3s observed on a single roll of a die( X = 1, for a roll of 3 and X = 0, for any other number), so E[X] = p

It can be shown that Var[X] = p(1 – p). Using this framework, the parameter p is identified

We can estimate p as precisely as we want given enough data on the roll of the die (given enough rolls of the die)

Assessing Data via Identification

‹#›

© 2019 McGraw-Hill Education.

The fact that p is identified follows directly from central limit theorem

Suppose the die is rolled N times. Define x1 as the observed values of X for the first roll, x2 for the second, and so on.

Then, define: = [ the sample mean for X, or equivalently, the portion of the N rolls that showed a 3

Given these definitions, the central limit theorem states that: ~ N(p, ) as N gets large

Assessing Data via Identification

‹#›

© 2019 McGraw-Hill Education.

Distribution of Mean of X for N =50 and N = 5,000

Assessing Data via Identification

‹#›

© 2019 McGraw-Hill Education.

Extrapolation and Interpolation

NOTE HOW THE VARIABLES “SALES” AND “PRICE” MOVE TOGETHER IN THE PRICE RANGE OF $210 TO $225 AND IN THE PRICE RANGE OF $275 TO $300

‹#›

© 2019 McGraw-Hill Education.

Suppose we want to know how Sales move with Prices in other price ranges

Interpolation involves drawing conclusions where there are “gaps” in the data

Data gap is any place where there are missing data for a variable over an interval of values, but data are not missing for at least some values on both ends of the interval

Extrapolation involves drawing conclusions beyond the extent of the data

Extrapolation and Interpolation

‹#›

© 2019 McGraw-Hill Education.

Must be considered when engaging in interpolation and/or extrapolation

The determining factor is whether the gap(s) in, or extend of, the data are due to random limitations in the sample or limitations in the population

If it is the former, there may be no identification problem

If it is the latter, then there is an identification problem that must be addressed

Identification Problems

‹#›

© 2019 McGraw-Hill Education.

Attempt to draw f(.) and g(.) without any mathematical formulas

WE ARE ATTEMPTING TO INTERPOLATE (FILL IN THE DATA GAP) AND ATTEMPTING TO EXTRAPOLATE (EXTEND BEYOND THE DATA’S RANGE).

Identification Problems

‹#›

© 2019 McGraw-Hill Education.

When interpolation or extrapolation is used to fill in gaps or limited extend of the data sample, but not the population, there is not an identification problem

When interpolation or extrapolation is used to fill gaps or limited extend of the population, there is an identification problem

No matter how much data is collected from the population, it will not help to draw any conclusions about what is happening in the unobserved range(s)

Identification Problems

‹#›

© 2019 McGraw-Hill Education.

Suppose you want to engage in interpolation and/or extrapolation when there exists an identification problem

For a general model of the data-generating process, where no assumptions are made about the determining function, we cannot sample more data from the population

There are two key approaches toward solving this type of identification problem:

Changes in the population

A functional form assumption

Remedies

‹#›

© 2019 McGraw-Hill Education.

Changing the population to alleviate an identification problem

A new singer has been promoting her music by selling physical copies of her music at various high schools.

She charges the same price to everyone and finds that the seniors buy the most often, freshman the least, and sophomores and juniors are in between

This tells her that her sales appear to be increasing by age of customers

She would like to extrapolate this relationship beyond just high school-aged kids

Using only data from high schools, she has an identification problem

Remedies: An Example

‹#›

© 2019 McGraw-Hill Education.

The figure illustrates possible ways to extrapolate past age 18, but there are no data to sort through the options.

A CLEAR OPTION TO THIS IDENTIFICATION PROBLEM WOULD BE TO TRY SELLING HER MUSIC AT COLLEGES AND COLLECT DATA ON HER SALES PERFORMANCE AMONG THIS GROUP.

THIS SIMPLE EXPANSION OF POPULATION WILL ALLEVIATE THE IDENTIFICATION PROBLEM.

Remedies

‹#›

© 2019 McGraw-Hill Education.

Imposing a functional form assumption to alleviate an identification problem

Standard practice is to assume a functional form of the determining function that applies for all relevant price levels

Assume a data-generating process with a linear functional form for the determining function: Salesi = α + βPricei + Ui

This assumption imposes the shape of the relationship between Sales and Price to be linear, but also dictates how to interpolate and/or extrapolate

Remedies

‹#›

© 2019 McGraw-Hill Education.

HERE, WE ARE ESTIMATING α AND β USING ONLY DATA WITH PRICE IN THE RANGES ($210, $225) AND ($275, $300).

WE ARE APPLYING THESE ESTIMATED VALUES ACROSS MANY OTHER PRICE LEVELS.

WE ARE USING THESE VALUES TO INTERPOLATE BETWEEN $225 AND $275 AND TO EXTRAPOLATE ALL THE WAY TO $350.

Regression Line for Rocking Chair Sales and Price Data

‹#›

© 2019 McGraw-Hill Education.

Another circumstance in which identification problems typically arise is when there is variable co-movement in the population

We use the broader term “co-movement” rather than correlation, since simple correlation alone do not encompass all the ways variables may move together in a population that result in identification problems

Variable Co-Movement

‹#›

© 2019 McGraw-Hill Education.

Variable Co-Movement

Three types of variable co-movement:

Perfect multicollinearity

Imperfect multicollinearity

Endogeneity

‹#›

© 2019 McGraw-Hill Education.

Consider the following data-generating process:

Yi = α + β1X1i +…+ βKXKi + Ui

Use regression analysis to estimate

We have assumed a functional form, so as long as there is some variation in there will not be identification problems stemming from voids in the data

There may be still be an identification problem when there is co-movement among the Xs and/or co-movement between one or more X and U

Variable Co-Movement

‹#›

© 2019 McGraw-Hill Education.

Perfect multicollinearity is a condition in which two or more independent variables have an exact linear relationship

If we can write there is perfect multicollinearity

Perfect multicollinearity in our model is equivalent to being able to express for all i in the population

Perfect multicollinearity implies a special type of correlation among two or more independent variables

Variable Co-Movement

‹#›

© 2019 McGraw-Hill Education.

Variable Co-Movement

Imperfect multicollinearity is a condition in which two or more independent variables have nearly an exact linear relationship

When this condition exists for a data-generating process, we can not express for all i in the population

Imperfect multicollinearity is equivalent to there being at least one semi-partial correlation that is “high”– nearly equal to 1

It is common to characterize a correlation above 0.8 as high

‹#›

© 2019 McGraw-Hill Education.

Variable Co-Movement

Endogeneity: in the context of identification problems involves co-movement between an independent variable(s) and the error term in a data-generating process

‹#›

© 2019 McGraw-Hill Education.

Perfect multicollinearity always leads to an identification problem in regression analysis

As an example, suppose, we believe that Sales of rocking chairs depends not only on price, but also on Distance from the designer’s location

We follow the data-generating process: Salesi = α + β1Pricei + β2Distancei + Ui

The population from which we are drawing suffers from perfect multicollinearity, creating an identification problem, particularly for β1 AND β2.

Identification Problems

‹#›

© 2019 McGraw-Hill Education.

The presence of perfect multicollinearity is clear, since we can write one independent variable as a linear function for another for every element in the population: Pricei = 200 + 0.04 × Distancei

The identification problem comes from the fact that we cannot separately estimate β1 and β2 – the marginal effect of Price and Distance on sales

The data-generating process becomes:

Salesi = α + β1(200 + 0.04 × Distancei)+ β2Distancei + Ui

Salesi = (α + β1200) + (0.04β1 + β2) Distancei + Ui

Perfect Multicollinearity

‹#›

© 2019 McGraw-Hill Education.

Three ways to detect perfect multicollinearity

A known linear relationship among two or more independent variables

Recognize misuse of dummy variables

Let the data reveal it

Perfect Multicollinearity

‹#›

© 2019 McGraw-Hill Education.

Imperfect multicollinearity does not cause an identification problem, it can create challenges with inference

imperfect multicollinearity can generate inflated p-values and confidence intervals, making it difficult to make any strong inductive arguments about population parameters

Because there is not an identification problem, these challenges go away with enough data

Imperfect Multicollinearity

‹#›

© 2019 McGraw-Hill Education.

To illustrate, imperfect multicollinearity, suppose, Price has a near-perfect linear relationship with Distance:

Pricei = 200 + 0.04 × Distancei + Vi,

where Vi contains other factors such as local fuel costs, etc.

A customer at a Distance of 2,000 miles might have a value for V of 3 and so face a Price of 200 + 0.04 × 2,000 + 3 = $283

A customer at a Distance of 400 miles might have a value for V of -2 and so face a Price of 200 + 0.04 × 400 ‒ 2 = $69

Price and Distance have imperfect multicollinearity

Imperfect Multicollinearity: An Example

‹#›

© 2019 McGraw-Hill Education.

Assume the following data-generating process:

Salesi = α + β1Pricei + β2Distancei + Ui

There is not perfect multicollinearity so we can get estimates of all the parameters when regressing Sales on Price and Distance

Imperfect Multicollinearity

‹#›

© 2019 McGraw-Hill Education.

Ways to check whether there is imperfect multicollinearity, and thus the possibility that this condition is inflating p-values and confidence intervals:

Calculate semi-partial correlations among independent variables and check whether they are close to 1

Variance inflation factor (VIF)

Imperfect Multicollinearity

‹#›

© 2019 McGraw-Hill Education.

Variation inflation factor (VIF) for an independent variable—say, —is equal to , where is the R-squared from regressing that independent variable (X1) on all other independent variables (X2,…,Xk) for a given determining function

A higher VIF for a given variable implies more noise (less certainity) in its coefficient estimator

VIF also tells us how much uncertainty this co-movement in the Xs is injecting into our estimators

Variation Inflation Factor (VIF)

‹#›

© 2019 McGraw-Hill Education.

Endogeneity can lead to estimators that are not consistent

Assume the following data-generating process:

Yi = α + β1X1i +…+ βKXKi + Ui

and there is a non-zero correlation between X1 and U

This correlation means 1 from a regression of Y on X1,…, XK need not be consistent

The inconsistency of 1 due to endogeneity amounts to endogeneity as an identification problem

Endogeneity as an Identification Problem

‹#›

© 2019 McGraw-Hill Education.

WE HAVE, 1 APPROACH A NUMBER C ≠ 1 AS THE SAMPLE GETS LARGE

Example of Inconsistent Estimator

‹#›

© 2019 McGraw-Hill Education.

The Effects of Variable Co-Movement on Identification

For the data-generating process Yi = α + β1X1i +…+ βKXKi + Ui : If there exists an exact linear relationship between at least two of the independent variables (Xs), defined as perfect multicollinearity, then there is an identification problem

In contract, if there is no exact linear relationship among the Xs, it is always possible to distinguish the effects of the independent variables on the outcome (Y) with any level of precision with sufficient data, even if some Xs exhibit imperfect multicollinearity

If there is correlation between any independent variable and the error term, defined as endogeneity, then there is an identification problem, no matter whether the correlation is via an exact linear relationship or not

‹#›

© 2019 McGraw-Hill Education.

For perfect multicollinearity

As long as our goal is to estimate the treatment effect and we have no particular interest in distinguishing the effects of controls, dropping one of the control variables contributing to perfect multicollinearity is an effective remedy

The only viable remedy when the treatment contributes to a perfect multicollinearity problem is to change the population from which you are sampling

Remedies for Identification Problems

‹#›

© 2019 McGraw-Hill Education.

Remedies for Identification Problems

For imperfect multicollinearity

If data are suffering from noisy estimates and VIF calculations suggest imperfect multicollinearity, the simple solution is to gather more data

If the imperfect multicollinearity involves only controls and there is no interest in estimating the effects of the controls per se, then collecting more data will not necessarily be worthwhile

‹#›

© 2019 McGraw-Hill Education.

Remedies for Identification Problems

For endogeneity

The only viable remedy is to change the population from which you are sampling

It does not matter whether the endogeneity involves the treatment or not

Options include: collecting controls, finding a proxy variable(s), finding an instrument(s), and/or transforming cross-sectional data to become a panel

‹#›

© 2019 McGraw-Hill Education.

Suppose we have assumed the following data-generating process: Yi = α + β1X1i +…+ βKXKi + Ui

Let X1 be the treatment and X2, … , XK be controls

Suppose that there is an omitted variable XK+1, that affects Y (and so is part of U) and is correlated with X1

The data generating process can be written as:

Yi = α + β1X1i +…+ βKXKi + βK+1XK+1i + Vi

Identification Damage Control: Signing the Bias

‹#›

© 2019 McGraw-Hill Education.

Let XK+1 = + X1i + …+ XKi be the estimated regression equation we get if we were to regress XK+1on X1, …, XK

Within this framework, define βK+1 × as the omitted variable bias

Omitted variable bias is the product of the effect of the omitted variable on the outcome (βK+1) and the (semi – partial) correlation between the omitted variable and the treatment ()

Identification Damage Control: Signing the Bias

‹#›

© 2019 McGraw-Hill Education.

Since we do not observe the omitted variable, we cannot estimate either of the components of omitted variable bias

We often can use theory to guide us with regard to the sign of each component.

The basic relationship is: sign(βK+1 × ) = sign(βK+1) × sign()

Identification Damage Control: Signing the Bias

‹#›

© 2019 McGraw-Hill Education.

Identification Damage Control: Signing the Bias

The four possibilities for the sign of the omitted variable bias is shown in the table below:

‹#›

© 2019 McGraw-Hill Education.