Input Modeling: Estimation, Testing and Correlation

Input modeling works on recorded data. It estimates the parameters of the data, tests whether a candidate distribution fits it, and checks whether two variables depend on each other. All six problems of Exercise 4 are solved step by step.

Objectives

• Estimate the sample mean and the sample variance from a frequency table.
• Check whether two jointly discrete random variables are independent.
• Estimate the parameters of lognormal data from its natural logarithms.
• Use the chi-square test to check whether data is Poisson distributed.
• Compute $E$, $Var$, $Cov$ and $Cor$ for two jointly continuous random variables.
• Measure the correlation between two recorded variables.

1. What this exercise does

Q1 and Q4 use the same data: Q4 tests the arrivals of Q1 and uses the mean computed in Q1.

2. Sample mean and variance

Q1: arrivals over 100 days

At a particular time of day, for a fixed time (10 minutes for example), the arrivals are recorded for 100 days and classified as in the table. Estimate the sample mean and variance.

Each row holds a value $m_j$ (arrivals per day) and its frequency $f_j$ (the number of days). Two extra columns prepare the two sums:

The sample mean:

$$\begin{aligned} \bar{X} &= \dfrac{\sum_{j=1}^{c} f_j m_j}{n} \\[6pt] \sum f_j m_j &= (0 \times 12) + (1 \times 10) \\ &\quad + \ldots + (11 \times 1) \\ &= 364 \\[6pt] \bar{X} &= \dfrac{364}{100} = 3.64 \end{aligned}$$

The sample variance:

$$\begin{aligned} S^2 &= \dfrac{\sum_{j=1}^{c} f_j m_j^2 - n \bar{X}^2}{n - 1} \\[6pt] &= \dfrac{2080 - 100 (3.64)^2}{100 - 1} \\[6pt] &= \dfrac{2080 - 1324.96}{99} \\[6pt] &= 7.63 \end{aligned}$$

Note on the exercise sheet. The sheet writes the sum as $(f_j m_j)^2$, and expands it as $(0 \times 12)^2 + (1 \times 10)^2 + \ldots$. Taken literally, that sum is 13684. The correct term is $f_j m_j^2$, which gives the 2080 that the sheet itself uses, so its answer 7.63 is right.

3. Dependence of jointly discrete variables

Q2: a joint probability function

$X$ and $Y$ are jointly discrete random variables with:

$$p(x, y) = \begin{cases} \dfrac{x + y}{30} & x = 0, 1, 2 \\ & \text{and } y = 0, 1, 2, 3 \\ 0 & \text{otherwise} \end{cases}$$

Check the dependence of $X$ and $Y$.

$X$ and $Y$ are independent if $P_X(x) \, P_Y(y) = p(x, y)$. Get $P_X(x)$ by summing over all $y$:

$$\begin{aligned} P_X(x) &= \sum_{\text{all } y} p(x, y) \\[6pt] &= \sum_{y=0}^{3} \dfrac{x + y}{30} \\[6pt] &= \dfrac{x}{30} + \dfrac{x + 1}{30} + \dfrac{x + 2}{30} + \dfrac{x + 3}{30} \\[6pt] &= \dfrac{4x + 6}{30} = \dfrac{2x + 3}{15} \end{aligned}$$

Get $P_Y(y)$ by summing over all $x$:

$$\begin{aligned} P_Y(y) &= \sum_{\text{all } x} p(x, y) \\[6pt] &= \sum_{x=0}^{2} \dfrac{x + y}{30} \\[6pt] &= \dfrac{y}{30} + \dfrac{y + 1}{30} + \dfrac{y + 2}{30} \\[6pt] &= \dfrac{3y + 3}{30} = \dfrac{y + 1}{10} \end{aligned}$$

Now multiply:

$$\begin{aligned} P_X(x) \, P_Y(y) &= \dfrac{(2x + 3)(y + 1)}{15 \times 10} \\[6pt] &= \dfrac{2xy + 3y + 2x + 3}{150} \\[6pt] &\neq p(x, y) \end{aligned}$$

Therefore $X$ and $Y$ are not independent. Three points show what the inequality means:

The two sides match at (1, 2), in fact at every point with x = 1. A match at one point proves nothing: independence needs a match at every point, and one point that fails is enough to show dependence.

4. Parameters of lognormal data

Q3: the rate of return on 10 investments

For a lognormal distribution, suppose you are given the rate of return on 10 investments: 18.8, 27.9, 21, 6.1, 37.4, 5, 22.9, 1, 3.1, 8.3.

To estimate the parameters of this lognormal data, take the natural logarithm of every value:

The estimated mean of the logarithms:

$$\begin{aligned} \sum x_i &= 2.9 + 3.3 + 3 + 1.8 + 3.6 \\ &\quad + 1.6 + 3.1 + 0 + 1.1 + 2.1 \\ &= 22.5 \\[6pt] \hat{\mu} &= \dfrac{22.5}{10} = 2.25 \end{aligned}$$

The estimated variance, where $x_i$ now means the logarithms and $\sum x_i^2 = 62.29$:

$$\begin{aligned} \hat{\sigma}^2 &= \dfrac{\sum_{i=1}^{10} x_i^2 - 10 \, \hat{\mu}^2}{10 - 1} \\[6pt] &= \dfrac{62.29 - 10 (2.25)^2}{9} \\[6pt] &= \dfrac{11.665}{9} = 1.30 \end{aligned}$$

Note on the exercise sheet. The sheet rounds $22.5 / 10$ to 2.3 and then uses 2.3 in the variance, which gives $(62.29 - 52.9) / 9 = 1.04$. Rounding the mean before squaring it changes the answer by a large amount. The correct values from the sheet's logarithms are $\hat{\mu} = 2.25$ and $\hat{\sigma}^2 = 1.30$. With the logarithms at full precision, they are $\hat{\mu} = 2.27$ and $\hat{\sigma}^2 = 1.32$.

5. Testing a distribution: the chi-square test

Q4: is the arrival data Poisson?

Use the chi-square test to check the following data as Poisson distributed:

This is the data of Q1, so n = 100, and the parameter is estimated by the mean: $\hat{\alpha} = 3.64$. The hypothesis $H_0$: the data is Poisson distributed. The symbol $\alpha$ has two meanings in this question: $\hat{\alpha} = 3.64$ is the Poisson parameter, and $\alpha = 0.05$ later is the level of significance.

The Poisson probability function gives the expected frequency:

$$f(x) = \begin{cases} \dfrac{e^{-\alpha} \alpha^x}{x!} & x = 0, 1, 2, \ldots \\ 0 & \text{otherwise} \end{cases}$$ $$E_i = n \, f(x)$$

For the first two values:

$$\begin{aligned} f(0) &= e^{-3.64} = 0.0263 \\ E &= 100 \times 0.0263 = 2.6 \\[6pt] f(1) &= 3.64 \, e^{-3.64} = 0.0956 \\ E &= 100 \times 0.0956 = 9.6 \end{aligned}$$

Note on the exercise sheet. The denominator of $f(x)$ on the sheet is written as $x$ with a subscript. It must be the factorial $x!$. The sheet also calls $f(x)$ a probability density function, although $x$ here is discrete. And it cuts 0.0956 to 0.095, which gives $E = 9.5$. Rounded correctly, $E = 9.6$.

Similarly, for all values:

Intervals of small values (for example $\le 5$) are combined with adjacent intervals. Here x = 0 joins x = 1, and x = 7 to x = 11 join into one interval, which leaves 7 intervals. Then compute $(O_i - E_i)^2 / E_i$ for each:

For example, for the first interval:

$$\dfrac{(22 - 12.2)^2}{12.2} = 7.87$$

Then the statistic is:

$$\begin{aligned} \chi_0^2 &= \sum_{i=1}^{7} \dfrac{(O_i - E_i)^2}{E_i} \\[6pt] &= 7.87 + 0.15 + 0.80 + 4.41 \\ &\quad + 2.57 + 0.26 + 11.63 \\ &= 27.69 \end{aligned}$$

From the chi-square table, read the value for $\alpha = 0.05$ and $k - s - 1$ degrees of freedom. Here k = 7 is the number of class intervals after combining the small ones, and s = 1 is the number of estimated parameters of the candidate distribution:

$$\begin{aligned} k - s - 1 &= 7 - 1 - 1 = 5 \\ \chi^2_{0.05, \, 5} &= 11.1 \end{aligned}$$

Since $\chi_0^2 = 27.69 > \chi^2_{0.05, 5} = 11.1$, $H_0$ is rejected: the data is not Poisson distributed.

Note on the exercise sheet. The sheet's term for x = 6 is 0.72, but $(7 - 8.5)^2 / 8.5 = 0.26$. With $E = 9.5$ it also has 12.1 and 8.1 for the first interval, where the correct values are 12.2 and 7.87. Its last term 11.62 should round to 11.63, because $88.36 / 7.6 = 11.626$. Its total is therefore 28.37, where the correct total is 27.69, or 27.63 when nothing is rounded before the sum. The decision does not change: each of these totals is above 11.1.

The chi-square table

The table on the sheet, with $\nu$ the degrees of freedom. Q4 reads row 5 in the column $\chi^2_{0.05}$.

Note on the exercise sheet. In the column $\chi^2_{0.005}$, the printed table on the sheet reads 14.96 in row 4 and 49.6 in row 25. The correct values are 14.86 and 46.9, and the table above uses them. Q4 does not read these rows.

6. Jointly continuous variables

Q5: dependence, covariance and correlation

$X$ and $Y$ are jointly continuous random variables with:

$$f(x, y) = \begin{cases} y - x & 0 \le x \le 1 \\ & \text{and } 1 \le y \le 2 \\ 0 & \text{otherwise} \end{cases}$$

(a) Check the dependence of $X$ and $Y$. (b) Compute $E(X)$, $Var(X)$, $E(Y)$, $Var(Y)$, $Cov(X, Y)$ and $Cor(X, Y)$.

Q5 (a): the dependence

Integrate over $y$ to get $f_X(x)$, and over $x$ to get $f_Y(y)$:

$$\begin{aligned} f_X(x) &= \int_{1}^{2} f(x, y) \, dy \\[6pt] &= \int_{1}^{2} (y - x) \, dy \\[6pt] &= \left[ \dfrac{y^2}{2} - xy \right]_{y=1}^{2} \\[6pt] &= \dfrac{3}{2} - x, \quad 0 \le x \le 1 \end{aligned}$$ $$\begin{aligned} f_Y(y) &= \int_{0}^{1} f(x, y) \, dx \\[6pt] &= \int_{0}^{1} (y - x) \, dx \\[6pt] &= \left[ yx - \dfrac{x^2}{2} \right]_{x=0}^{1} \\[6pt] &= y - \dfrac{1}{2}, \quad 1 \le y \le 2 \end{aligned}$$ $$\begin{aligned} f_X(x) \, f_Y(y) &= \left( \dfrac{3}{2} - x \right) \left( y - \dfrac{1}{2} \right) \\[6pt] &\neq f(x, y) \end{aligned}$$

So $X$ and $Y$ are not independent.

Q5 (b): the means and the variances

$$\begin{aligned} E(X) &= \int_{0}^{1} x \left( \dfrac{3}{2} - x \right) dx \\[6pt] &= \left[ \dfrac{3}{2} \cdot \dfrac{x^2}{2} - \dfrac{x^3}{3} \right]_{0}^{1} \\[6pt] &= \dfrac{3}{4} - \dfrac{1}{3} = \dfrac{5}{12} \end{aligned}$$ $$\begin{aligned} E(X^2) &= \int_{0}^{1} x^2 \left( \dfrac{3}{2} - x \right) dx \\[6pt] &= \left[ \dfrac{3}{2} \cdot \dfrac{x^3}{3} - \dfrac{x^4}{4} \right]_{0}^{1} \\[6pt] &= \dfrac{3}{6} - \dfrac{1}{4} = \dfrac{1}{4} \end{aligned}$$ $$\begin{aligned} Var(X) &= E(X^2) - [E(X)]^2 \\[6pt] &= \dfrac{1}{4} - \left( \dfrac{5}{12} \right)^2 \\[6pt] &= \dfrac{36 - 25}{144} = \dfrac{11}{144} \end{aligned}$$ $$\begin{aligned} E(Y) &= \int_{1}^{2} y \left( y - \dfrac{1}{2} \right) dy \\[6pt] &= \left[ \dfrac{y^3}{3} - \dfrac{1}{2} \cdot \dfrac{y^2}{2} \right]_{1}^{2} \\[6pt] &= \dfrac{7}{3} - \dfrac{3}{4} = \dfrac{19}{12} \end{aligned}$$ $$\begin{aligned} E(Y^2) &= \int_{1}^{2} y^2 \left( y - \dfrac{1}{2} \right) dy \\[6pt] &= \left[ \dfrac{y^4}{4} - \dfrac{1}{2} \cdot \dfrac{y^3}{3} \right]_{1}^{2} \\[6pt] &= \dfrac{15}{4} - \dfrac{7}{6} = \dfrac{31}{12} \end{aligned}$$ $$\begin{aligned} Var(Y) &= E(Y^2) - [E(Y)]^2 \\[6pt] &= \dfrac{31}{12} - \left( \dfrac{19}{12} \right)^2 \\[6pt] &= \dfrac{372 - 361}{144} = \dfrac{11}{144} \end{aligned}$$

Q5 (b): the covariance and the correlation

$$\begin{aligned} E(XY) &= \int_{x=0}^{1} \int_{y=1}^{2} xy \, f(x, y) \, dy \, dx \\[6pt] &= \int_{x=0}^{1} \int_{y=1}^{2} xy (y - x) \, dy \, dx \\[6pt] &= \int_{x=0}^{1} \left[ \dfrac{x y^3}{3} - \dfrac{x^2 y^2}{2} \right]_{y=1}^{2} dx \\[6pt] &= \int_{x=0}^{1} \left( \dfrac{7x}{3} - \dfrac{3x^2}{2} \right) dx \\[6pt] &= \left[ \dfrac{7x^2}{6} - \dfrac{3x^3}{6} \right]_{0}^{1} \\[6pt] &= \dfrac{7}{6} - \dfrac{1}{2} = \dfrac{4}{6} = \dfrac{2}{3} \end{aligned}$$

Note on the exercise sheet. The second line of this integral on the sheet is written with $xy(x - y)$. Since $f(x, y) = y - x$, the correct form is $xy(y - x)$. With $x - y$ the result would be $-2/3$. The sheet's next line follows from $y - x$, so its value 2/3 is correct. The sheet also writes this expected value as $E(x, y)$; it is the expected value of the product $XY$. It also writes the double integral with dx dy, although its next line integrates over y first, so this page writes dy dx.

$$\begin{aligned} Cov(X, Y) &= E(XY) - E(X) E(Y) \\[6pt] &= \dfrac{2}{3} - \left( \dfrac{5}{12} \cdot \dfrac{19}{12} \right) \\[6pt] &= \dfrac{96 - 95}{144} = \dfrac{1}{144} \end{aligned}$$ $$\begin{aligned} Cor(X, Y) &= \dfrac{Cov(X, Y)}{\hat{\sigma}_1 \hat{\sigma}_2} \\[6pt] &= \dfrac{Cov(X, Y)}{\sqrt{Var(X) \, Var(Y)}} \\[6pt] &= \dfrac{1 / 144}{\sqrt{\dfrac{11}{144} \cdot \dfrac{11}{144}}} \\[6pt] &= \dfrac{1 / 144}{11 / 144} = \dfrac{1}{11} \end{aligned}$$

7. Correlation from recorded data

Q6: lead time and demand

In an inventory model, $X_1$ is the average lead time to deliver an order, in months, and $X_2$ is the annual demand, for the last 10 years:

Check if there is any correlation between the lead time and the demand. The sample mean and the sample variance are:

$$\mu = \dfrac{\sum_{i=1}^{n} x_i}{n}, \qquad S^2 = \dfrac{\sum_{i=1}^{n} x_i^2 - n \mu^2}{n - 1}$$

The two means:

$$\begin{aligned} \mu_1 &= \dfrac{6.5 + 4.3 + \cdots + 6.3}{10} \\[6pt] &= \dfrac{61.4}{10} = 6.14 \\[6pt] \mu_2 &= \dfrac{103 + 83 + \cdots + 96}{10} \\[6pt] &= \dfrac{1018}{10} = 101.8 \end{aligned}$$

The lead time variance and its estimated standard deviation:

$$\begin{aligned} \sum x_i^2 &= (6.5)^2 + (4.3)^2 + \cdots + (6.3)^2 \\ &= 386.44 \\[6pt] S_1^2 &= \dfrac{386.44 - 10 (6.14)^2}{9} \\[6pt] &= \dfrac{386.44 - 376.996}{9} \\[6pt] &= 1.05 \\[6pt] \hat{\sigma}_1 &= \sqrt{1.05} = 1.025 \end{aligned}$$

The annual demand variance and its estimated standard deviation:

$$\begin{aligned} \sum x_i^2 &= (103)^2 + (83)^2 + \cdots + (96)^2 \\ &= 104520 \\[6pt] S_2^2 &= \dfrac{104520 - 10 (101.8)^2}{9} \\[6pt] &= \dfrac{104520 - 103632.4}{9} \\[6pt] &= 98.6 \\[6pt] \hat{\sigma}_2 &= \sqrt{98.6} = 9.93 \end{aligned}$$

The covariance:

$$\widehat{Cov}(X_1, X_2) = \dfrac{\sum_{j=1}^{n} X_{1j} X_{2j} - n \, \mu_1 \mu_2}{n - 1}$$ $$\begin{aligned} \sum X_{1j} X_{2j} &= (103 \times 6.5) + (83 \times 4.3) \\ &\quad + \cdots + (96 \times 6.3) \\ &= 6328.5 \\[6pt] n \, \mu_1 \mu_2 &= 10 \times 6.14 \times 101.8 \\ &= 6250.52 \\[6pt] \widehat{Cov} &= \dfrac{6328.5 - 6250.52}{9} = 8.66 \end{aligned}$$

The correlation coefficient:

$$\begin{aligned} \hat{\rho} &= \dfrac{\widehat{Cov}(X_1, X_2)}{\hat{\sigma}_1 \hat{\sigma}_2} \\[6pt] &= \dfrac{8.66}{1.025 \times 9.93} = 0.85 \end{aligned}$$

The value is close to 1, so $X_1$ and $X_2$ are very strongly positively correlated.

Note on the exercise sheet. The sheet adds "if $X_1 < \mu_1$ then $X_2 < \mu_2$, and if $X_1 > \mu_1$ then $X_2 > \mu_2$". This is a tendency and not a rule: 8 of the 10 years follow it, and the years (5.8, 106) and (6.3, 96) do not.

8. Common mistakes

1. Squaring the whole product. The variance term is $f_j m_j^2$. In Q1, $(f_j m_j)^2$ would add up to 13684, not 2080.
2. Rounding the mean before squaring it. In Q3, 2.3 in place of 2.25 turns the variance 1.30 into 1.04.
3. Stopping at one matching point. In Q2 the two sides match at every point with x = 1, and $X$ and $Y$ are still not independent.
4. Forgetting the factorial. The Poisson denominator is $x!$. It decides every expected frequency from x = 2 upward.
5. Wrong degrees of freedom. Count k after combining the small intervals, and subtract s for the estimated parameter: 7 - 1 - 1 = 5 in Q4, not 12 - 1 = 11.
6. Reversing the decision. A statistic larger than the table value means $H_0$ is rejected.

9. Practice

Solve each problem, then check the answers at the end of this page. All of them use the data and the table of this exercise.

1. For Q2, compare $p(x, y)$ with $P_X(x) \, P_Y(y)$ at the point (2, 0).
2. For Q4, compute $f(2)$ and the expected frequency $E$ from the formula.
3. For Q4, repeat the decision at $\alpha = 0.01$.
4. For Q5, integrate $f(x, y)$ over its whole range, and give the result.
5. Repeat Q3 with the logarithms rounded to two decimals, and compute $\hat{\mu}$ and $\hat{\sigma}^2$.

Key takeaways

• From a frequency table, the sample mean uses $\sum f_j m_j$ and the sample variance uses $\sum f_j m_j^2$.
• Two variables are independent only if the product of their distributions equals the joint function at every point.
• For lognormal data, take the natural logarithms first, then compute their mean and variance.
• Chi-square: compute $E_i = n \, f(x)$, combine the small intervals, sum $(O_i - E_i)^2 / E_i$, and compare with the table at $k - s - 1$ degrees of freedom.
• $Var(X) = E(X^2) - [E(X)]^2$ and $Cov(X, Y) = E(XY) - E(X) E(Y)$.
• $Cor = Cov / (\sigma_1 \sigma_2)$: a value close to 1 means a strong positive correlation.

Answers

Practice 1

$p(2, 0) = 2 / 30 = 10 / 150$. $P_X(2) = 7 / 15$ and $P_Y(0) = 1 / 10$, so the product is $7 / 150$. The two values do not match, as expected for dependent variables.

Practice 2

$$\begin{aligned} f(2) &= \dfrac{e^{-3.64} (3.64)^2}{2!} = 0.1739 \\[6pt] E &= 100 \times 0.1739 = 17.4 \end{aligned}$$

This is the value in the table of Q4.

Practice 3

The degrees of freedom stay at 5, and the table gives $\chi^2_{0.01, 5} = 15.1$. Since $27.69 > 15.1$, $H_0$ is still rejected.

Practice 4

$$\begin{aligned} \int_{0}^{1} \int_{1}^{2} (y - x) \, dy \, dx &= \int_{0}^{1} \left( \dfrac{3}{2} - x \right) dx \\[6pt] &= \left[ \dfrac{3x}{2} - \dfrac{x^2}{2} \right]_{0}^{1} \\[6pt] &= \dfrac{3}{2} - \dfrac{1}{2} = 1 \end{aligned}$$

Practice 5

The logarithms are 2.93, 3.33, 3.04, 1.81, 3.62, 1.61, 3.13, 0, 1.13, 2.12. Their sum is 22.72, so $\hat{\mu} = 2.272$. The sum of their squares is 63.46, so:

$$\begin{aligned} \hat{\sigma}^2 &= \dfrac{63.46 - 10 (2.272)^2}{9} \\[6pt] &= \dfrac{63.46 - 51.62}{9} \\[6pt] &= 1.32 \end{aligned}$$