Read This First
If this page feels abrupt, start here
These links provide the wider frame, earlier distinction, or branch map that makes the current page easier to enter.
-
The Professional Application of Rationality
Start here if the current page feels compressed: The Professional Application of Rationality gives the broader frame before the argument narrows into the present pressure.
-
Rational Thought Branch Guide
If this page feels abrupt, start with the Rational Thought branch guide so the wider map is visible before the close reading begins.
Read This Next
If the page clicked, continue here
These are not just nearby pages. They are the strongest next moves if you want the pressure of this page to keep unfolding.
-
What is Game Theory?
What is Game Theory? keeps the same branch pressure in view but turns it from a different angle.
Prompt 1: Provide a short but clear definition of perverse incentives?
A perverse incentive is a reward structure that makes the wrong behavior look rational from inside the system.
A perverse incentive appears when a system rewards behavior that undermines the system's own stated goal. The people responding to the incentive may not be irrational or malicious; they may simply be adapting sensibly to the payoff structure placed in front of them.
That is why the concept matters so much in public policy, management, education, criminal justice, and online platforms. A rule can be well-intended at the level of slogan while still pushing participants toward gaming, concealment, short-termism, or visible compliance that defeats the deeper purpose.
The real lesson is not 'people are bad.' The lesson is that behavior follows incentives more faithfully than mission statements. If the metric is crude, the system may teach smart people to optimize the wrong thing.
A simple example is enough to make the idea vivid: if a school rewards teachers mainly for test scores, some teachers may start teaching to the test or pushing weak students aside. The people involved may still claim allegiance to education, but the structure is already training them to chase the proxy.
That is why perverse incentives are often invisible to the people who built them. Everyone may sincerely endorse the stated mission while the actual reward structure quietly teaches another lesson. Good intentions are fully compatible with bad system design.
A good definition should therefore keep intention and outcome separate. Designers may want one result, while the structure they built quietly rewards another.
- Goal mismatch: The announced aim of the system differs from the behavior its rewards and penalties actually encourage.
- Adaptive behavior: Participants often respond quite rationally to the incentive even when the overall outcome is harmful.
- Metric danger: Once a number becomes the target, people may optimize the number rather than the underlying good.
- Design lesson: Moral sincerity at the top does not protect a system from bad incentives built into its machinery.
- Reader lesson: To find a perverse incentive, ask what behavior the structure makes attractive, not what the policy says it wants.
Prompt 2: Provide 10 diverse stories from history of perverse incentives.
Historical examples matter because perverse incentives keep returning in different costumes.
A page of historical examples should not read like a cabinet of curiosities. The point is to show recurrence. Different eras and institutions invent different policies, but the same structural mistake keeps appearing: reward the proxy, and people may sacrifice the real goal to improve the proxy.
That is why pirate suppression, bounty systems, testing regimes, performance bonuses, and policing quotas can belong on the same page. On the surface they look unrelated. Underneath they share a common logic: the incentive teaches participants how to win the game while quietly worsening the thing the game was supposed to improve.
Historical range matters because it prevents the reader from treating perverse incentives as a niche quirk. They are a permanent design risk wherever human beings respond creatively to rewards, punishments, and measurable targets.
A good treatment should therefore extract the repeating pattern from the stories. Otherwise the reader leaves with anecdotes instead of a reusable diagnostic tool.
The British government, aiming to reduce the cobra population, offered a bounty for every dead cobra. This led to people breeding cobras to kill and submit for the bounty. When the scheme was abandoned, breeders released the now-worthless snakes, increasing the cobra population.
Similarly, the French colonial government in Vietnam offered a reward for each rat tail presented as proof of killing a rat to combat bubonic plague. People began cutting off tails and releasing rats to breed, so they could continue to collect bounties.
In 1958, Mao Zedong declared sparrows to be pests and initiated a campaign to eliminate them, resulting in mass killings of sparrows. The lack of sparrows led to a surge in crop-eating insects, contributing to the Great Chinese Famine.
In 1935, cane toads were introduced into Australia to control cane beetle populations. However, the toads couldn’t jump high enough to eat the beetles, which live on top of sugar cane stalks. The toads became an invasive species, causing ecological problems without controlling the beetles.
To reduce pollution, Mexico City implemented a program where cars were banned from the city one day a week based on license plate numbers. Instead of reducing the number of cars, it led to people buying older, more polluting cars to use on their off days, increasing overall emissions.
Named after Barbra Streisand, who in 2003 tried to suppress photographs of her home, leading to widespread distribution of the images. Efforts to hide, remove, or censor information can result in increased publicity.
The Jordanian government granted exclusive publication rights to a small team of scholars in the 1950s. This monopoly led to significant delays in the release of the texts, encouraging black market sales and limiting academic study.
To protect white employment, the government implemented job reservation policies, restricting certain jobs to white people only. This led businesses to mechanize operations, reducing overall employment and harming the very group the policy intended to protect.
Introduced in 1696, this tax was based on the number of windows in a house as a proxy for the owner’s wealth. To reduce tax liability, people bricked up windows, leading to poor ventilation and health problems in buildings, an outcome contrary to the tax’s intention to be a fair wealth indicator.
To reduce the rat population, Parisians offered a bounty for dead rats. This backfired as people started breeding rats to claim the reward, leading to a bigger problem!
In an attempt to reduce coal smoke pollution, London implemented a tax on coal use. However, people switched to burning cheaper, but smokier alternatives, worsening the fog.
Chairman Mao encouraged the elimination of sparrows, believing they ate grain. People killed millions, but with their predators gone, insect populations exploded, causing far greater crop damage.
To reduce venomous cobra populations, the Indian government offered bounties for dead cobras. This led to people breeding cobras to claim the reward, then releasing them!
Boeing offered bonuses to engineers based on the number of planes produced. This resulted in cutting corners on safety features, leading to several deadly accidents.
Texas offered financial incentives to doctors for performing certain procedures. This led to unnecessary surgeries for profit, not patient needs.
Intended to curb drug use, this policy led to increased incarceration for minor offenses, straining the justice system and disproportionately impacting minorities.
Financial institutions were incentivized to sell risky subprime mortgages, leading to a housing bubble and the 2008 financial crisis.
Online platforms reward content based on clicks, not quality. This incentivizes sensational and misleading headlines, eroding trust in information.
- Proxy failure: Systems often reward an easily measured stand-in for the real goal, then get more of the stand-in and less of the goal.
- Gaming pattern: Once the incentive is visible, clever participants learn how to satisfy the metric without serving the mission.
- Cross-domain recurrence: The same structural error appears in empires, classrooms, markets, bureaucracies, and digital platforms.
- Historical payoff: The examples matter because they teach a pattern the reader can spot again elsewhere.
- Reader lesson: Diverse stories become useful when they reveal one recurring mechanism rather than ten disconnected oddities.
Prompt 3: What kinds of principles and heuristics can we employ to reduce the chances of introducing perverse incentives when creating policies?
Policy design gets wiser when it asks how smart people will game the rule before the rule goes live.
The best protection against perverse incentives is anticipatory humility. Before a policy is launched, designers should ask not only 'what behavior do we want?' but also 'how could a reasonably clever person satisfy this rule while betraying its purpose?' That second question is where many good intentions fail.
Reducing perverse incentives usually requires multiple safeguards: mixed metrics rather than one brittle target, periodic review, room for qualitative judgment, and feedback from people affected by the rule. A single-number regime often invites clever compliance and mission drift.
It also helps to assume that visibility changes behavior. Once participants know what is counted, they start adapting to the count. That is not corruption in every case; it is ordinary human responsiveness. Good policy design plans for it.
So the key heuristics are not mysterious. Stress-test the proxy, imagine the exploit, reward the underlying goal where possible, and revise quickly when the system starts teaching the wrong lesson.
Before implementation, conduct comprehensive assessments to predict potential outcomes and side effects of a policy. This includes evaluating historical precedents and consulting a wide range of experts to foresee unintended consequences.
Roll out policies gradually in phases or pilot programs before full implementation. This allows for monitoring and adjustment based on real-world feedback and data, minimizing the risk of widespread perverse incentives.
Establish mechanisms for continuous feedback from those affected by the policy. Be prepared to make adjustments and amendments to the policy as unintended consequences are identified.
Carefully design incentives so that they directly encourage the behaviors that lead to the desired outcomes, rather than just intermediate metrics that can be easily gamed.
Engage a diverse group of stakeholders in the policy design process, including those who will be directly affected. This can uncover potential perverse incentives that policymakers might not anticipate.
Policies that focus too narrowly on specific metrics can lead to gaming the system. Instead, use broader, more comprehensive measures of success to avoid incentivizing undesired behaviors.
Ensure that the goals and mechanisms of the policy are communicated clearly and transparently to prevent misunderstandings that could lead to perverse incentives.
Complex policies with numerous conditions and exceptions are more likely to lead to unintended outcomes. Simplifying policies can help ensure that the incentives align more closely with the desired behaviors.
Implement robust systems to monitor outcomes and evaluate the policy’s effectiveness over time. This should include indicators to detect perverse incentives early.
Evaluate the ethical implications and equity impacts of a policy to ensure it does not disproportionately benefit or harm certain groups, which could lead to broader societal perverse incentives.
Imagine everyone involved is trying to “win” within the system. How could they exploit it for personal gain, even if unintended?
Identify all parties affected by the policy (citizens, businesses, government agencies) and consider their potential responses.
While outputs (arrests made, test scores) are easy to measure, focus on the desired long-term outcomes (public safety, well-rounded education).
Consider how the policy might incentivize behavior in unforeseen circumstances as situations evolve.
Define goals explicitly and choose metrics that truly reflect progress towards them.
Build in mechanisms to track the policy’s effectiveness and identify unintended consequences early.
Test policies on a small scale before full implementation to identify and address perverse incentives early.
Be prepared to adjust the policy based on real-world data and feedback from those affected.
- Exploit rehearsal: Before implementation, actively imagine how the rule could be satisfied in spiritless or harmful ways.
- Mixed metrics: Use more than one indicator so participants cannot cheaply optimize a single brittle target.
- Human review: Preserve some judgment from people who can notice when the metric looks good while the reality worsens.
- Fast revision: Incentive systems should be monitored and adjusted rather than treated as set-and-forget moral machinery.
- Reader lesson: The safest policy question is often not 'does this sound good?' but 'how will this be gamed?'
Prompt 4: How can we identify and quarantine bad actors who are trying to game the system?
To catch bad actors, systems need cross-checks that make gaming costly without punishing ordinary participation.
Identifying bad actors starts with a distinction that many systems blur: some failures come from flawed incentives, while others come from deliberate exploitation. If that difference is ignored, institutions either over-punish ordinary users who are merely responding to the system as designed, or under-punish genuine gamers who have learned how to hide inside normal-looking behavior.
Good detection therefore relies on layered signals rather than a single red flag. Sudden statistical anomalies, repeated edge-case behavior, impossible performance spikes, networked coordination, and suspicious timing patterns can all suggest gaming. None is decisive by itself, but together they can mark a pattern that deserves scrutiny.
Quarantining bad actors should also be proportionate. Random audits, temporary holds, appeals processes, graduated sanctions, and human review are usually wiser than instant permanent exclusion based on one suspicious metric. Otherwise the cure becomes its own perverse incentive, teaching innocent people to fear participation while teaching sophisticated abusers how to stay just beneath the threshold.
The broader lesson is that enforcement itself must be designed with incentive-awareness. A system should make exploitation harder, honest participation safer, and diagnosis more careful than theatrical.
Use advanced data analytics to monitor behavior patterns and detect anomalies that indicate someone is trying to game the system. This could include unusual transactions, patterns of behavior that match known fraudulent activities, or other red flags.
Implement machine learning algorithms that can learn from past incidents of gaming the system to better predict and identify future attempts. These tools can adapt over time as bad actors evolve their strategies.
Strengthen verification processes to ensure that users are who they claim to be. This can include multi-factor authentication, biometric verification, and background checks for high-stakes environments.
Leverage the power of the community by allowing users to report suspicious behavior. This not only helps in identifying bad actors but also engages the community in maintaining the integrity of the system.
Clearly communicate the rules of the system and the consequences for attempting to game it. Knowing that there are stringent checks in place and that penalties will be enforced can deter bad actors.
Conduct regular audits and reviews of transactions, user activities, and system vulnerabilities. This helps in early detection of potential gaming and provides insights for improving security measures.
Collaborate with other organizations, agencies, or platforms to share information about known bad actors and emerging strategies for gaming the system. Collective defense strategies can be more effective than going it alone.
Develop systems that can dynamically respond to detected threats by limiting functionality, quarantining accounts, or implementing additional verification processes in real-time.
Ensure that there are robust legal and policy frameworks that support the identification, prosecution, and punishment of individuals or entities attempting to game the system. This includes having clear policies on privacy and data protection.
Ensure that the tools and algorithms used for monitoring and enforcement are used ethically, with transparency about how decisions are made, particularly if they impact access to services or impose penalties.
Use data analysis to identify unusual activity patterns that deviate from expected behavior. This could involve sudden spikes in claims, rapid completion rates, or geographically concentrated activity suggesting organized manipulation.
Develop scoring systems that assign a risk level to individuals or activities based on pre-defined criteria. These criteria could include inconsistencies in application data, past history of abuse, or connections to known bad actors.
Implement stronger verification processes beyond basic credentials. This could involve incorporating phone confirmations, biometrics, or additional documentation.
Conduct regular, random audits of high-risk applications or suspicious activity to verify legitimacy.
Invest in personnel trained to identify red flags and investigate potential fraud.
Develop partnerships with law enforcement and other stakeholders to share information and build a comprehensive picture of bad actors.
In certain cases, temporary or permanent denial of access might be necessary to prevent further exploitation.
Restrict access to specific features or benefits for high-risk individuals, allowing them to participate but limiting their ability to manipulate the system.
- Design-versus-abuse distinction: Separate ordinary adaptation to a flawed rule from deliberate efforts to manipulate the system unfairly.
- Pattern detection: Look for clusters of anomalies, timing irregularities, impossible gains, or repeated edge-case exploitation rather than a single suspicious event.
- Proportionate response: Temporary quarantines, audits, and appeals are often more rational than irreversible punishment at first contact.
- False-positive caution: Overzealous enforcement can create a new incentive problem by frightening legitimate users or rewarding performative compliance.
- Reader lesson: The best anti-gaming system is careful enough to catch abuse without turning caution into collateral damage.
What ties this page together.
A useful path through this branch is practical. Ask what mistake the page helps detect, what habit it trains, and what kind of disagreement it makes less confused.
The danger is performative rationality: naming fallacies, probabilities, or methods while using them as badges rather than tools for better judgment.
Keep what Perverse Incentives is being used to explain, the objection that would change the answer, and a borderline case where the idea strains in the same frame. That is what shows what the page is claiming, where it gets tested, and what would have to change if the claim is right.
Read this page as part of the wider Rational Thought branch: the prompts point inward to the topic, but they also point outward to neighboring questions that keep the topic honest.
For a companion resource on calibration, credence, and structured rational judgment, see Credencing.com.
- What is a perverse incentive?
- What unintended consequence did the Great Sparrow Campaign in China have on agriculture?
- How did the introduction of cane toads to Australia backfire?
- Which distinction inside Perverse Incentives is easiest to miss when the topic is explained too quickly?
- What is the strongest charitable reading of this topic, and what is the strongest criticism?
Deep Understanding Quiz Check your understanding of Perverse Incentives
This quiz checks whether the main distinctions and cautions on the page are clear. Choose an answer, read the feedback, and click the question text if you want to reset that item.
Future Branches
Where this page naturally expands
Nearby pages in the same branch include What is Game Theory?; those links are not decorative, but suggested continuations where the pressure of this page becomes sharper, stranger, or more usefully contested.