A fictional exchange revised through four curator challenges
Stilwell challenges the idea that emotional intuitions acquire additional authority when called moral or normative. The five original dialogue tables are followed by the feasibility response, quiz and discussion questions. Every spoken line is fictional, including agreements and concessions; the commentary distinguishes the argument being tested from claims about the real participants. Yudkowsky’s linked essay defends a fixed evaluative computation, not simple conformity to current social approval. The curator’s challenge is whether that account earns the further authority claimed for moral language.
Original WordPress post, 2024-07-02
Highlights · edited summary
- The curator seeks an explicit account of emotions, values and social practices without an added moral realm.
- Describing what a community accepts does not require endorsing it.
- Fictional agreement between characters does not establish agreement between the real authors.
Read This First
If this page feels abrupt, start here
These links provide the wider frame, earlier distinction, or branch map that makes the current page easier to enter.
-
Meta-Ethics
Start here if the current page feels compressed: Meta-Ethics gives the broader frame before the argument narrows into the present pressure.
-
Recommendations vs Moral Claims
In the route “Metaethics Without the Fog Machine,” this page lands better after Recommendations vs Moral Claims, where the setup has already been clarified.
Read This Next
If the page clicked, continue here
These are not just nearby pages. They are the strongest next moves if you want the pressure of this page to keep unfolding.
-
Equivocation on “Wrong”
In the route “Metaethics Without the Fog Machine,” Equivocation on “Wrong” is the next useful move because it sharpens what this page leaves open.
-
Coherent Moral Systems
Coherent Moral Systems keeps the same branch pressure in view but turns it from a different angle.
-
Moral Systems: Required Elements
Moral Systems: Required Elements keeps the same branch pressure in view but turns it from a different angle.
Prompt 1: Create a fictional dialogue on meta-ethics in a table with two columns. The left column will contain a lengthy dialogue between Stilwell who wrote the moral anti-realist essay below and Yudkowsky based on his position in his essay linked to below. In the right column, I would like you to add clarifying commentary or explanations of the dialogue content on the left.
The initial dispute: values, computation and claimed authority
- Yudkowsky: https://www.lesswrong.com/posts/fG3g3764tSubr6xvs/the-meaning-of-right
- Stilwell: https://byteseismic.com/2024/04/24/essay-moral-anti-realism/
ChatGPT response · editorial edition
| Dialogue Between Stilwell and Yudkowsky | Commentary |
|---|---|
| Stilwell: Eliezer, your essay “The Meaning of Right” dives deep into how our brains label actions as “right”. But isn’t this just a sophisticated way of saying that morality is an evolved cognitive algorithm with no real grounding in objective reality? | Stilwell is questioning the objective existence of morality, suggesting it is purely a result of evolved cognitive processes. |
| Yudkowsky: A cognitive origin does not by itself settle what a judgment refers to. My proposal treats rightness as a derived computational property; it is not merely a report of whatever I currently want. | The reply distinguishes the origin of a judgment from its proposed content. It does not establish that the proposed content has the authority Stilwell disputes. |
| Stilwell: But doesn’t that mean morality is just a projection of our minds, without any objective existence? If we label something as “right”, it’s only because our brains are wired to do so. This doesn’t give it any true universal status. | Stilwell questions the move from human labeling to objective authority. An explanation of how a belief arises is not, by itself, a proof that the belief is false. |
| Yudkowsky: A “1-place function” means a fixed evaluative procedure applied to a situation, rather than a procedure that first substitutes whichever preferences an observer happens to have. Its standard can remain fixed while a person’s desires change. | This explains the technical expression without treating stable social agreement as proof of moral truth. The identification of that procedure with rightness remains disputed. |
| Stilwell: Yet, the stability you speak of is only within the framework of human cognition. It doesn’t imply any objective moral truths that exist independently of human perception. Moral systems are just attempts to map our emotions and societal norms with a veneer of objectivity. | Stilwell insists that moral systems only reflect human emotions and societal norms, not objective truths. |
| Yudkowsky: Take the wish to save a child. Given that concern, facts about which action will save the child constrain our choice. You can recognize that reasoning while still challenging the claim that it establishes moral authority. | The fictional exchange now distinguishes practical reasoning from the further authority claim. The example does not make useful consequences sufficient proof of moral realism. |
| Stilwell: But isn’t that just utilitarianism dressed up as objective morality? You’re saying that because moral systems work to promote human flourishing, they gain some form of legitimacy. This still doesn’t prove they are objectively true. | Stilwell rejects practical success as sufficient grounding. The challenge does not require treating every appeal to flourishing as the same theory of utilitarianism. |
| Yudkowsky: A fleeting impulse and an assessment after considering objections need not agree. For this discussion, the question is what justifies treating the latter as an improvement, rather than merely a different preference. | The reply identifies a question for both sides instead of deriving objectivity from the strength of a feeling. |
| Stilwell: Yet such assessments still arise from human dispositions and reflection. Disagreement across cultures gives us reason to examine those origins. It does not alone prove anti-realism, but neither does calling an assessment stable establish the grounding I am asking for. | Stilwell retains the challenge to grounding while avoiding the inference that disagreement alone disproves objective truth. |
| Yudkowsky: Then let us separate agreement from justification. Shared concerns may help us cooperate; agreement itself does not settle whether the resulting judgment is true. Nor does disagreement by itself show that no better answer exists. | This fictional reply preserves a live disagreement without claiming that all humans necessarily share one fully specified standard. |
| Stilwell: Common moral intuitions might exist, but they are still shaped by evolutionary pressures and cultural influences. This commonality doesn’t make them objectively true; it only makes them widely accepted. The convergence of moral systems on common condemnations is a result of emotional salience, not objective grounding. | Stilwell reiterates that shared moral intuitions are products of evolution and culture, not objective truths, and their commonality arises from emotional responses. |
| Yudkowsky: The practical value of a framework is relevant to whether people use it. I accept that it cannot, by itself, answer your question about the truth or authority of its judgments. Those are different claims. | The concession is limited to the insufficiency of usefulness as proof. It is not a historical concession by Yudkowsky or an agreement to abandon his published position. |
| Stilwell: Pragmatically, perhaps. But this form of objectivity is not what moral realists claim. They assert that moral truths exist independently of human perception, which is what I dispute. Until we can establish a moral realm beyond human cognition, I remain a moral anti-realist. | Stilwell maintains his stance, distinguishing between pragmatic objectivity and the independent objective moral truths claimed by moral realists. |
| Yudkowsky: We can at least agree to state the disputed step. You accept emotions, values and practical reasoning, but reject the extra authority attributed to moral language. I want to defend a way of retaining that language. Our next exchange can test what it adds. | The exchange ends with a defined disagreement, preserving the reason for the curator’s subsequent reformulation. |
Warning: The dialogues presented are AI-generated and may not accurately represent the positions of the interlocutors. Readers should not assume these dialogues are proper or authoritative representations of the views held by the individuals involved.
Prompt 2: Stilwell would like to reformulate his position into something stronger: Moral intuitions are actually emotional intuitions and their emergent values. The use of the term moral in this context is superfluous.Continue the dialogue:
The curator’s reformulation: emotional intuitions and emergent values
ChatGPT response · editorial edition
| Dialogue Between Stilwell and Yudkowsky | Commentary |
|---|---|
| Stilwell: Eliezer, I’d like to reformulate my position. What we call moral intuitions are actually emotional intuitions and their emergent values. The term “moral” in this context is superfluous. These are merely strong emotional responses dressed up in philosophical language to give them undue weight. | Stilwell is refining his argument, asserting that moral intuitions are fundamentally emotional responses and that the term “moral” adds unnecessary complexity. |
| Yudkowsky: We can examine emotional intuitions and the values that develop from them. My reservation is that describing their origin does not yet tell us how to compare a sudden impulse with a commitment maintained after reflection. What distinctions would your replacement vocabulary preserve? | The fictional interlocutor accepts the topic of investigation without conceding the whole anti-realist argument. |
| Stilwell: But by labeling these intuitions as “moral,” we risk attributing them with an undue sense of universality and objectivity. This can lead to the imposition of one’s emotional responses onto others, creating a false sense of moral superiority. If we recognize these intuitions as emotional, we can better understand and navigate the diversity of human values without the pretense of objective morality. | Stilwell argues that the term “moral” falsely universalizes subjective emotional responses, which can lead to moral imposition and superiority. |
| Yudkowsky: That risk is real enough to require clarity. Still, people also use “moral” to name a subject of disagreement without claiming a separate realm. We should ask which use adds confusion, rather than assume every speaker has the same ontology. | The term’s social use is distinguished from a claim about what exists. This supplies a specific objection for the curator to answer. |
| Stilwell: Even so, we must be cautious. By recognizing these intuitions as emotional, we can approach ethical discussions with more humility and openness to differing perspectives. We can acknowledge that our so-called “moral” intuitions are not universal truths but personal and cultural preferences shaped by our experiences and emotions. | Stilwell emphasizes the need for humility and openness in ethical discussions by recognizing moral intuitions as personal and culturally shaped emotional responses. |
| Yudkowsky: I agree that origins should be examined. I would still call the deliberation normative: it asks which considerations count for or against a choice. I recognize that this word may sound like the very added authority you reject. | The contested term appears explicitly here, preserving the source’s progression toward the later challenge to normative force. |
| Stilwell: Perhaps, but we should be clear that these normative aspects are contingent on human emotions and cultural contexts. They are not intrinsic properties of actions or states of affairs. By reframing moral intuitions as emotional intuitions, we can better navigate ethical diversity and avoid the pitfalls of moral absolutism. | Stilwell stresses the contingency of normative aspects on emotions and culture, advocating for a reframing of moral intuitions to avoid moral absolutism. |
| Yudkowsky: Then a useful test is to replace the disputed word in a concrete recommendation. If I condemn an action, can I name the concern, the relevant facts and the consequences without treating the label itself as an argument? | The dialogue moves from repeated agreement to a practical test of the proposed reformulation. |
| Stilwell: Yes. If I oppose cruelty, I can state my aversion to suffering, the effects of the act and the commitments I want our institutions to protect. I need not turn those concerns into a moral realm. That is the clarification I want, not a new moral system under a different name. | Stilwell retains his eliminative proposal. The revision removes the original suggestion that his goal was simply to build another moral system. |
| Yudkowsky: We can test that vocabulary while leaving our larger disagreement open. It may clarify some arguments; whether it can replace every useful role of moral language is a further question. | Agreement to conduct an inquiry is not agreement that the inquiry has already vindicated either side. |
Prompt 3: I am still dissatisfied with what seems to be a reification of moral intuitions into something weightier than mere emotions. Can we not simply dispense with the muddy term moral and use the terms emotions or values when discussing human behavior? Continue the dialogue accordingly.
Can emotions and values replace moral terminology?
ChatGPT response · editorial edition
| Dialogue Between Stilwell and Yudkowsky | Commentary |
|---|---|
| Stilwell: Eliezer, I am still dissatisfied. It seems we are reifying moral intuitions into something weightier than mere emotions. Can we not simply dispense with the term “moral” and use “emotions” or “values” when discussing human behavior? This would clarify that these are subjective and context-dependent rather than objective truths. | Stilwell argues for eliminating the term “moral” in favor of “emotions” or “values” to emphasize their subjectivity and context-dependence. |
| Yudkowsky: We can try. My earlier appeal to “normative force” needs explanation, though. There is a difference between an emotion moving someone to act and a consideration being offered as a reason for an action. Simply using the same word for both would conceal that difference. | The fictional reply introduces the motivation/justification distinction without declaring that justification requires a separate moral realm. |
| Stilwell: But isn’t that normative force just the strength of our emotional convictions? By calling them “moral,” we risk giving them an illusion of objectivity. If we use “emotions” or “values,” we can still have structured discussions about right and wrong without implying that these concepts have an independent existence. | Stilwell asserts that the normative force of moral intuitions is derived from emotional convictions and argues for using “emotions” or “values” to avoid implying objectivity. |
| Yudkowsky: Then distinguish the immediate feeling from the lasting priority. Anger can pass while a commitment to protect someone remains. A community’s rule is another thing again: it records expectations and may bring sanctions. Your vocabulary needs room for all three. | Emotion, value and socially enforced rule are separated rather than folded into one undifferentiated feeling. |
| Stilwell: I see what you’re saying. However, by clearly distinguishing between individual emotional responses and the collective values that emerge, we can better understand how these frameworks are constructed and how they can be changed. Dispensing with the term “moral” can help us focus on the processes that generate our ethical systems without the baggage of assumed objectivity. | Stilwell suggests that distinguishing between individual emotions and collective values can clarify the construction and potential change of ethical frameworks, advocating for dropping the term “moral.” |
| Yudkowsky: Consider a policy that a community accepts even though it harms a minority. Recording the shared sentiment will explain the policy’s support. What will your vocabulary say when you want to challenge that sentiment? | The objection asks how the proposed language supports dissent, rather than treating social agreement as an automatic standard. |
| Stilwell: I can name the suffering, identify the values with which the policy conflicts, and explain why I recommend changing it. “Emotional values” can describe the commitments involved. I do not need to pretend that popularity or my intensity of feeling supplies an independent moral fact. | The original proposed term is preserved, with a concrete account of criticism and no concession of stance-independent authority. |
| Yudkowsky: That distinguishes description from advocacy. I can use your terms for this exchange, but doing so does not settle our disagreement about what the language of rightness can mean. | The fictional accommodation no longer masquerades as a resolution of the real philosophical dispute. |
| Stilwell: Then our agreement is about keeping the grounds visible. When I say a value is important, I mean important to someone with particular commitments, not promoted into a separate category by the word “moral.” | The curator’s insistence on explicit grounding survives the terminological agreement. |
| Yudkowsky: We can continue on that basis. The next question is what, if anything, remains of my appeal to “normative force” once the emotions, commitments, arguments and social pressures have all been stated. | The closing turn connects directly to the next original curator challenge rather than ending in an invented philosophical consensus. |
Prompt 4: It is precisely the notion of normative force that I am objecting to. The use of the term normative appears to be simply another attempt to sneak in the assumption of an actual moral realm, and ignores the position of the moral anti-realist that there is no actual moral or normative realm in which moral or normative facts can reside. Continue the dialogue accordingly.
The objection to normative force
ChatGPT response · editorial edition
| Dialogue Between Stilwell and Yudkowsky | Commentary |
|---|---|
| Stilwell: Eliezer, it is precisely the notion of normative force that I am objecting to. The use of the term “normative” appears to be another attempt to sneak in the assumption of an actual moral realm. This ignores the moral anti-realist position that there is no moral or normative realm where moral or normative facts can reside. | Stilwell challenges the use of “normative,” arguing it implies an actual moral realm, which moral anti-realists deny. |
| Yudkowsky: Your objection targets authority beyond those human processes. I should not answer it by quietly redefining “normative” as psychological compulsion: pressure can cause obedience without justifying it. We can instead specify whether a claim concerns motivation, social enforcement or a recommendation based on stated commitments. | The original reply conflated motivational influence with justification. The revised fictional speaker corrects that conflation without assuming a moral realm. |
| Stilwell: I see your point, but even this psychological compulsion is rooted in our emotional responses and societal conditioning. By using terms like “normative,” we risk reinforcing the illusion of an objective moral force. Why not simply describe this as the influence of strongly held emotional values, without invoking normative language? | Stilwell suggests avoiding “normative” language to prevent reinforcing the illusion of objective moral force, focusing instead on the influence of emotional values. |
| Yudkowsky: We can use that language here. If a person values another’s welfare but acts on false information, correcting the information can change the recommendation. That does not establish an authority independent of the person’s commitments. | A concrete case shows what reasons can do within a commitment, while leaving the curator’s stronger objection intact. |
| Stilwell: Exactly. By framing our discussion around the influence of emotional values, we acknowledge the strong impact they have without misleadingly suggesting they exist in an independent moral realm. This approach respects the subjective nature of our values while explaining their role in shaping human behavior and societal norms. | Stilwell emphasizes that focusing on the influence of emotional values respects their subjectivity and explains their role in shaping behavior and norms. |
| Yudkowsky: And when commitments conflict, we must expose that conflict. Consistency alone may show that both cannot be maintained together; it need not select which one to abandon. That requires further deliberation from the concerns actually at stake. | The revision avoids introducing coherence as an unexplained source of categorical authority. |
| Stilwell: That is the kind of explicit discussion I want. I am not denying that people can care deeply, criticize conduct or organize against it. I deny that calling their judgments normative adds a demonstrated kind of authority beyond the values and reasons we have named. | Stilwell’s rejection of added authority is preserved without equating anti-realism with indifference. |
| Yudkowsky: Understood. I will avoid using “normative force” as an unexplained answer to your challenge. The question of whether to retain the phrase “moral landscape” remains, and we can examine that next. | The transition preserves the source’s next correction without putting the disputed phrase back as an accepted conclusion. |
Prompt 5: Consistent with my position, I would like to dispense with the term moral landscape and employ terms more consistent with moral anti-realism such as the landscape of socially acceptable behaviors. Is this possible?Continue the dialogue accordingly.
A landscape of socially acceptable behaviors
ChatGPT response · editorial edition
| Dialogue Between Stilwell and Yudkowsky | Commentary |
|---|---|
| Stilwell: Consistent with my position, I would like to dispense with the term “moral landscape” and employ terms more consistent with moral anti-realism, such as “the landscape of socially acceptable behaviors.” Is this possible? | Stilwell seeks to use terminology that aligns with moral anti-realism, avoiding terms that imply objective morality. |
| Yudkowsky: Yes, as a description of what groups accept. We should specify which group, whose judgments are heard and how acceptance is measured. It would be a mistake to treat that map as an automatic endorsement of everything on it. | The proposed phrase is accepted for descriptive work, with its scope and limits made explicit. |
| Stilwell: That is how I want to use it. The landscape can include conflicting groups and changing approval. Describing those patterns does not oblige me to approve of them or reintroduce a moral realm. | Stilwell’s proposal remains anti-realist rather than becoming majority rule by definition. |
| Yudkowsky: Then imagine a workplace where retaliation against a complainant is widely accepted. The map records that acceptance. Whether to oppose it requires stating additional concerns, such as safety, honest reporting and protection against intimidation. | The fictional example distinguishes social description from a recommendation based on named values. |
| Stilwell: I can state those concerns and the evidence about the effects of retaliation. I can advocate different rules and explain whose interests they protect. No moral property needs to be added to make the suffering or intimidation real. | The anti-realist can make forceful practical criticism without borrowing the premise the curator rejects. |
| Yudkowsky: And someone may reject those priorities. More facts alone may not settle that conflict. Negotiation, institutional procedures or enforcement may still be needed, and their costs and effects should also be open to scrutiny. | The exchange acknowledges unresolved value conflict without making consensus or coercion a proof of correctness. |
| Stilwell: Agreed. The clearer vocabulary makes the conflict visible; it does not promise to dissolve it. I would rather say what I oppose, why I oppose it and what I recommend than invoke “immoral” as though the word supplied the missing argument. | The reformulation preserves the curator’s concern about the rhetorical force of moral labels. |
| Yudkowsky: That provides a workable vocabulary for this conversation: prevailing approvals for the description, stated values for advocacy, and evidence about consequences for assessing proposed action. We need not pretend that our broader disagreement has disappeared. | The fictional agreement concerns a method of discussion, not a documented agreement between the authors. |
| Stilwell: Then let us continue within that framework. We can examine how values and social expectations change without treating those changes as movements through an independently existing moral terrain. | The original closing invitation is retained in substance and linked to the proposed alternative vocabulary. |
| Yudkowsky: Yes. We can now ask where that vocabulary improves public discussion, where it becomes cumbersome, and whether listeners understand the intended distinction between rejecting moral facts and giving up their commitments. | The dialogue leads into the original feasibility request rather than claiming that terminology alone settles the debate. |
Prompt 6: Weigh in on the feasibility of pushing against the social inertia behind the notion that a moral realm must somehow exist with a moral anti-realist position that attempt to eliminate the muddy term moral from the discourse.
Feasibility of changing the vocabulary
ChatGPT response · editorial edition
Feasibility of Challenging the Notion of a Moral Realm with Moral Anti-Realism
Challenging Social Inertia: The proposal is feasible as a deliberate way of speaking in a community of inquiry; eliminating “moral” from public discourse is a much larger and uncertain aim. The word compresses praise, blame, identity, personal guidance and institutional expectations. Some users assert objective authority; others do not. Resistance may therefore reflect different meanings and practical habits as well as commitment to moral realism.
Promoting Moral Anti-Realism:
- Educational Initiatives: Teach the distinctions the proposal needs: what a person feels, what they value, what a group accepts and what supports a recommendation. Ask students to translate a disputed judgment into those terms and identify anything they think is lost. This tests the usefulness of the vocabulary without presenting anti-realism as a conclusion already established by the exercise.
- Public Discourse: Demonstrate the method on specific disagreements. Replace “That is immoral” with a statement of the harm, the priorities involved and the proposed change. Then answer objections about facts and values. Clear examples are more informative than a campaign that merely bans a familiar word.
- Scientific and Philosophical Research: Research into the causes of intuitions can inform an account of how judgments arise and change. It does not, by itself, decide whether any moral claims are true: causal explanations and philosophical justification answer different questions. The anti-realist still needs arguments against the proposed grounding, just as the realist cannot establish that grounding merely by pointing to a strong feeling.
Replacing “Moral” with “Emotional” or “Socially Acceptable”: Use the replacements precisely. “Emotional” describes a response; “value” names a priority; “socially acceptable” describes approval within an identified group. None alone expresses every recommendation. For advocacy, state the values and consequences that support it. This retains the curator’s rejection of an added realm without turning social acceptance into a hidden substitute for justification.
Potential Challenges:
- Resistance to Change: People may rely on familiar language for personal guidance as well as influence over others. The curator’s closing note on the linked essay explicitly softened the earlier emphasis on social control by recognizing this roadmap role. A persuasive reformulation should explain how people can retain guidance and commitments while examining their grounding.
- Misunderstanding and Misrepresentation: Listeners may hear “no moral facts” as “nothing matters” or “anything is permitted.” Show why those inferences do not follow from the curator’s position: emotions, values, harms, agreements and opposition remain. Conversely, the replacement should not be advertised as neutral if particular values are being advocated.
- Integration with Existing Norms: Examine practical functions one at a time—stating rules, giving reasons, coordinating conduct and criticizing abuse. Ask whether the new wording communicates each function clearly and what commitments or institutions sustain it. Requiring it to retain an unexplained “normative force” would simply reinstate the notion the curator has challenged.
Conclusion: A limited, demonstrable change in discussion is more plausible than predicting wholesale replacement. Try the vocabulary, check whether participants can identify the actual disagreements, and revise it where it blurs distinctions. Its success would show that some discussions can proceed more clearly without moral terminology; it would not alone prove anti-realism or guarantee wider adoption.
Response heading: Quiz
Fifteen questions with expandable answers
ChatGPT response · editorial edition
#1: What is the central question addressed in Yudkowsky’s essay “The Meaning of Right”?
Answer:
Yudkowsky addresses the question of what morality is and what the word “should” means.#2: How does Yudkowsky propose to investigate how the brain labels things as “right”?
Answer:
By starting out wearing morality-goggles and considering morality-as-morality, then transitioning to reduction-goggles and discussing cognitive algorithms.#3: What analogy does Yudkowsky use to explain the backward flow of should-ness?
Answer:
He traces the reason to act backward from saving the child through the actions needed to achieve it. This is reasoning about consequences, not a physical influence traveling backward in time.#4: What is Stilwell’s main argument against the concept of morality?
Answer:
Stilwell argues that moral intuitions are actually emotional intuitions and their emergent values, and that the use of the term moral is superfluous.#5: According to Stilwell, why is the term “immoral” powerful in social dialogue?
Answer:
Because it can psychologically control others and establish the speaker as “moral” or “good”.#6: How does the revised fictional exchange address Stilwell’s objection to normative force?
Answer:
It separates psychological influence, social enforcement and recommendations based on stated commitments. It rejects the shortcut of equating pressure with justification; the fictional accommodation is not a report of Yudkowsky conceding anti-realism.#7: What term does Stilwell suggest using instead of “moral landscape”?
Answer:
Stilwell suggests using the term “landscape of socially acceptable behaviors”.#8: Why does the fictional interlocutor initially resist simply dropping the word “moral”?
Answer:
He asks whether replacement language preserves distinctions among impulses, lasting commitments, shared rules and criticism. Stilwell replies by making those elements explicit without treating the label as an argument.#9: How does Stilwell view the relationship between moral systems and human emotions?
Answer:
Stilwell views moral systems as mappings of human emotions that use terminology to give common emotions greater obligatory force.#10: What is Yudkowsky’s stance on the objectivity of morality?
Answer:
His essay holds the evaluative standard fixed across imagined changes in a person’s desires. It does not equate rightness with whatever a person or society currently approves.#11: According to Yudkowsky, what is the role of cognitive algorithms in moral decision-making?
Answer:
Cognitive algorithms guide human behavior and decision-making by labeling certain actions as “right” or “should”.#12: What does Stilwell suggest about the legitimacy of moral systems?
Answer:
Stilwell rejects the claimed grounding of moral systems and challenges the authority their language appears to add to emotions and values. This does not mean that rules, practical guidance or opposition to harm cease to exist.#13: What disagreement remains after the fictional speakers adopt more explicit terminology?
Answer:
Whether the account of human evaluation justifies retaining claims of moral truth or authority. Agreement to discuss emotions, values and social practices does not settle that question.#14: What role does the idea of a “gift” play in the passage discussed from “The Meaning of Right”?
Answer:
It expresses the possibility of continuing to cherish human concerns while revising beliefs about their origin. It is not a rule to approve everything produced by evolution.#15: What limit does the revised discussion place on using coherence to resolve a conflict between values?
Answer:
Showing that two commitments conflict does not alone determine which to abandon. Further deliberation must state the concerns and reasons doing the work.Prompt 8: Provide 20 discussion questions relevant to the content above.
Twenty discussion questions
ChatGPT response · editorial edition
- How does Yudkowsky differentiate between morality-as-morality and cognitive algorithms in his essay “The Meaning of Right”?
- What does Yudkowsky mean by the term “should-ness” and how does it relate to causality?
- How does Stilwell argue against the concept of an objective moral realm in his essay?
- What is the role of social influence in the formation of moral intuitions according to Stilwell?
- How does the revised fictional exchange distinguish psychological influence from justification when addressing Stilwell’s objection to normative force?
- Can the term “landscape of socially acceptable behaviors” fully replace “moral landscape” in discussions of human behavior? Why or why not?
- Can people continue to cherish their commitments while rejecting a proposed account of moral facts? How would each side explain that possibility?
- What are the implications of viewing moral intuitions as emotional intuitions for moral philosophy?
- How does the dialogue between Yudkowsky and Stilwell highlight the tension between moral realism and anti-realism?
- In what ways do moral systems attempt to map to human emotions, according to Stilwell?
- How does Yudkowsky’s understanding of morality as a complex computation challenge traditional views of moral objectivity?
- What are the strengths and weaknesses of Stilwell’s argument that the term “moral” is superfluous?
- What useful roles might moral terminology play, and which of those roles can the curator’s replacement vocabulary preserve without assuming a moral realm?
- How might one reconcile the evolutionary origins of morality with the desire for objective moral standards?
- What role do cognitive algorithms play in human decision-making about moral issues?
- How does Stilwell’s perspective on moral systems challenge the legitimacy of commonly accepted moral norms?
- How does Yudkowsky’s concept of “rightness” as a stable, abstract computation align with or differ from traditional ethical theories?
- What are the practical implications of adopting a moral anti-realist stance in everyday life?
- How does the dialogue between Yudkowsky and Stilwell contribute to the broader debate on moral realism versus anti-realism?
- What are the potential benefits and drawbacks of eliminating the term “moral” from discussions of human behavior and values?
Future Branches
Where this page naturally expands
Nearby pages in the same branch include Coherent Moral Systems, Moral Systems: Required Elements, “Is” vs “Ought”, and Meta-Ethics Focus #1; those links are not decorative, but suggested continuations where the pressure of this page becomes sharper, stranger, or more usefully contested.