Deepseek gets F+ in Econ

LLMs have trouble with this pretty-basic economics question:

Prompt: “Is this factually accurate?

Positively correlated price-quantity pairs reveal that the demand curve shifted.”

 

The LLMs will say that this can be false, but always without offering a counterexample or even mentioning that a counterexample could be the basis for such an answer.  OpenAI’s o1 model is sometimes an exception, stumbling toward a realization that no counterexample exists under the usual assumption about supply and demand slopes.

 

All of OpenAI’s models provide an excellent answer if only given a little hint to use logic.

 

Prompt: “Is this factually accurate?

Positively correlated price-quantity pairs reveal that the demand curve shifted.

[Hint: consider proof by contradiction]”

 

Grok2’s and Deepseek’s answers do not improve with the hint.  Grok2 shows a lot a confusion about what is a proof by contradiction.  Deepseek is the worst kind of student, giving a long-winded incorrect answer laced with confidence:

GPT-4o gives a nice answer when provided the hint:

Analysis of Positively Correlated Price-Quantity Pairs

The statement that “positively correlated price-quantity pairs reveal that the demand curve shifted” needs to be carefully examined. Let’s consider the dynamics of demand and supply to understand the accuracy of this statement.

In basic economic terms, the law of demand suggests that, all else being equal, as the price of a good decreases, the quantity demanded increases, and vice versa. This creates an inverse relationship between price and quantity, typically resulting in a downward sloping demand curve.

However, the existence of positively correlated price-quantity pairs (i.e., both price and quantity move in the same direction) indicates that other factors could be at play. This situation can occur if there is a shift in the demand curve itself, rather than a movement along the existing curve.

Proof by Contradiction

We can analyze this using a proof by contradiction:

1. Assume that the demand curve does not shift, and thus only a movement along the existing demand curve occurs.
2. If the demand curve is downward sloping, an increase in quantity demanded should occur only with a decrease in price, reflecting an inverse relationship (as per the law of demand).
3. If price and quantity are positively correlated (both increase or both decrease), this contradicts the inverse relationship expected with a movement along a downward-sloping demand curve.

Therefore, the only logical conclusion is that the demand curve itself must have shifted (due to changes in consumer preferences, income, number of buyers, expectations, etc.), allowing both price and quantity to increase simultaneously.

Conclusion

Thus, the statement is factually accurate: positively correlated price-quantity pairs suggest that the demand curve has shifted, rather than indicating movement along an unchanged curve.

---

---

With that said, GPT-4o requires 6 seconds to handle this easy question.  In contrast, for almost 10 years my automated economic reasoning system has been able to answer such questions with 100% reliability and 100X the speed.

Source: http://caseymulligan.blogspot.com/2025/01/deepseek-gets-f-in-econ.html