Understanding from Machine Learning Models.Emily Sullivan -2022 -British Journal for the Philosophy of Science 73 (1):109-133.detailsSimple idealized models seem to provide more understanding than opaque, complex, and hyper-realistic models. However, an increasing number of scientists are going in the opposite direction by utilizing opaque machine learning models to make predictions and draw inferences, suggesting that scientists are opting for models that have less potential for understanding. Are scientists trading understanding for some other epistemic or pragmatic good when they choose a machine learning model? Or are the assumptions behind why minimal models provide understanding misguided? In (...) this paper, using the case of deep neural networks, I argue that it is not the complexity or black box nature of a model that limits how much understanding the model provides. Instead, it is a lack of scientific and empirical evidence supporting the link that connects a model to the target phenomenon that primarily prohibits understanding. (shrink)
Reliability in Machine Learning.Thomas Grote,Konstantin Genin &Emily Sullivan -2024 -Philosophy Compass 19 (5):e12974.detailsIssues of reliability are claiming center-stage in the epistemology of machine learning. This paper unifies different branches in the literature and points to promising research directions, whilst also providing an accessible introduction to key concepts in statistics and machine learning – as far as they are concerned with reliability.
Vulnerability in Social Epistemic Networks.Emily Sullivan,Max Sondag,Ignaz Rutter,Wouter Meulemans,Scott Cunningham,Bettina Speckmann &Mark Alfano -2020 -International Journal of Philosophical Studies 28 (5):1-23.detailsSocial epistemologists should be well-equipped to explain and evaluate the growing vulnerabilities associated with filter bubbles, echo chambers, and group polarization in social media. However, almost all social epistemology has been built for social contexts that involve merely a speaker-hearer dyad. Filter bubbles, echo chambers, and group polarization all presuppose much larger and more complex network structures. In this paper, we lay the groundwork for a properly social epistemology that gives the role and structure of networks their due. In particular, (...) we formally define epistemic constructs that quantify the structural epistemic position of each node within an interconnected network. We argue for the epistemic value of a structure that we call the (m,k)-observer. We then present empirical evidence that (m,k)-observers are rare in social media discussions of controversial topics, which suggests that people suffer from serious problems of epistemic vulnerability. We conclude by arguing that social epistemologists and computer scientists should work together to develop minimal interventions that improve the structure of epistemic networks. (shrink)
Inductive Risk, Understanding, and Opaque Machine Learning Models.Emily Sullivan -2022 -Philosophy of Science 89 (5):1065-1074.detailsUnder what conditions does machine learning (ML) model opacity inhibit the possibility of explaining and understanding phenomena? In this article, I argue that nonepistemic values give shape to the ML opacity problem even if we keep researcher interests fixed. Treating ML models as an instance of doing model-based science to explain and understand phenomena reveals that there is (i) an external opacity problem, where the presence of inductive risk imposes higher standards on externally validating models, and (ii) an internal opacity (...) problem, where greater inductive risk demands a higher level of transparency regarding the inferences the model makes. (shrink)
Idealizations and Understanding: Much Ado About Nothing?Emily Sullivan &Kareem Khalifa -2019 -Australasian Journal of Philosophy 97 (4):673-689.detailsBecause idealizations frequently advance scientific understanding, many claim that falsehoods play an epistemic role. In this paper, we argue that these positions greatly overstate idealiza...
Do ML models represent their targets?Emily Sullivan -forthcoming -Philosophy of Science.detailsI argue that ML models used in science function as highly idealized toy models. If we treat ML models as a type of highly idealized toy model, then we can deploy standard representational and epistemic strategies from the toy model literature to explain why ML models can still provide epistemic success despite their lack of similarity to their targets.
SIDEs: Separating Idealization from Deceptive ‘Explanations’ in xAI.Emily Sullivan -forthcoming -Proceedings of the 2024 Acm Conference on Fairness, Accountability, and Transparency.detailsExplainable AI (xAI) methods are important for establishing trust in using black-box models. However, recent criticism has mounted against current xAI methods that they disagree, are necessarily false, and can be manipulated, which has started to undermine the deployment of black-box models. Rudin (2019) goes so far as to say that we should stop using black-box models altogether in high-stakes cases because xAI explanations ‘must be wrong’. However, strict fidelity to the truth is historically not a desideratum in science. Idealizations (...) – the intentional distortions introduced to scientific theories and models – are commonplace in the natural sciences and are seen as a successful scientific tool. Thus, it is not falsehood qua falsehood that is the issue. In this paper, I outline the need for xAI research to engage in idealization evaluation. Drawing on the use of idealizations in the natural sciences and philosophy of science, I introduce a novel framework for evaluating whether xAI methods engage in successful idealizations or deceptive explanations (SIDEs). SIDEs evaluates whether the limitations of xAI methods, and the distortions that they introduce, can be part of a successful idealization or are indeed deceptive distortions as critics suggest. I discuss the role that existing research can play in idealization evaluation and where innovation is necessary. Through a qualitative analysis we find that leading feature importance methods and counterfactual explanations are subject to idealization failure and suggest remedies for ameliorating idealization failure. (shrink)
Understanding: not know-how.Emily Sullivan -2018 -Philosophical Studies 175 (1):221-240.detailsThere is considerable agreement among epistemologists that certain abilities are constitutive of understanding-why. These abilities include: constructing explanations, drawing conclusions, and answering questions. This agreement has led epistemologists to conclude that understanding is a kind of know-how. However, in this paper, I argue that the abilities constitutive of understanding are the same kind of cognitive abilities that we find in ordinary cases of knowledge-that and not the kind of practical abilities associated with know-how. I argue for this by disambiguating between (...) different senses of abilities that are too often lumped together. As a consequence, non-reductionists about understanding—those that claim that understanding-why is not reducible to knowledge-that—need to find another way to motivate the view. In the end, the fact that abilities are constitutive of understanding-why does not give us reason to conclude that understanding is a kind of know-how. (shrink)
Negative Epistemic Exemplars.Mark Alfano &Emily Sullivan -2019 - In Benjamin R. Sherman & Stacey Goguen,Overcoming Epistemic Injustice: Social and Psychological Perspectives. London: Rowman & Littlefield International.detailsIn this chapter, we address the roles that exemplars might play in a comprehensive response to epistemic injustice. Fricker defines epistemic injustices as harms people suffer specifically in their capacity as (potential) knowers. We focus on testimonial epistemic injustice, which occurs when someone’s assertoric speech acts are systematically met with either too little or too much credence by a biased audience. Fricker recommends a virtuetheoretic response: people who do not suffer from biases should try to maintain their disposition towards naive (...) testimonial justice, and those who find themselves already biased should cultivate corrective testimonial justice by systematically adjusting their credence in testimony up or down depending on whether they are hearing from someone whom they may be biased against or in favor of. We doubt that the prominent admirationemulation model of exemplarism will be much use in this connection, so we propose two ways of learning from negative exemplars to better conduct one’s epistemic affairs. In the admirationemulation model, both the identification of what a virtue is and the cultivation of virtues identified thusly proceed through the admiration of virtuous exemplars. We show that this model has serious flaws and argue for two alternatives: the envyagonism model and the ambivalenceavoidance model. (shrink)
Universality caused: the case of renormalization group explanation.Emily Sullivan -2019 -European Journal for Philosophy of Science 9 (3):36.detailsRecently, many have argued that there are certain kinds of abstract mathematical explanations that are noncausal. In particular, the irrelevancy approach suggests that abstracting away irrelevant causal details can leave us with a noncausal explanation. In this paper, I argue that the common example of Renormalization Group explanations of universality used to motivate the irrelevancy approach deserves more critical attention. I argue that the reasons given by those who hold up RG as noncausal do not stand up to critical scrutiny. (...) As a result, the irrelevancy approach and the line between casual and noncausal explanation deserves more scrutiny. (shrink)
Online trust and distrust.Mark Alfano &Emily Sullivan -2021 - In Michael Hannon & Jeroen de Ridder,The Routledge Handbook of Political Epistemology. New York: Routledge.detailsTrust makes cooperation possible. It enables us to learn from others and at a distance. It makes democratic deliberation possible. But it also makes us vulnerable: when we place our trust in another’s word, we are liable to be deceived—sometimes intentionally, sometimes unintentionally. Our evolved mechanisms for deciding whom to trust and whom to distrust mostly rely on face-to-face interactions with people whose reputation we can both access and influence. Online, these mechanisms are largely useless, and the institutions that might (...) supplant them need to have their own trustworthiness verified. Currently, those institutions are mostly corporations such as Facebook and Twitter, which have checkered track records at best. To make matters worse, the social media sector is a natural monopoly, and companies like Facebook have shown that they are willing to use their market power unscrupulously. For these reasons, we argue that social media should be treated like other natural monopolies: it should either be nationalized, highly regulated, or broken up through antitrust legal actions. (shrink)
Humility in networks.Mark Alfano &Emily Sullivan -2020 - In Mark Alfano, Michael Patrick Lynch & Alessandra Tanesini,The Routledge Handbook of the Philosophy of Humility. New York, NY: Routledge.detailsWhat do humility, intellectual humility, and open-mindedness mean in the context of inter-group conflict? We spend most of our time with ingroup members, such as family, friends, and colleagues. Yet our biggest disagreements —— about practical, moral, and epistemic matters —— are likely to be with those who do not belong to our ingroup. An attitude of humility towards the former might be difficult to integrate with a corresponding attitude of humility towards the latter, leading to smug tribalism that masquerades (...) as genuine virtue. These potentially conflicting priorities have recently come to the fore because “tribal epistemology” has so thoroughly infected political and social discourse. Most research on these dispositions focuses on individual traits and dyadic peer-disagreement, with little attention to group membership or inter-group conflict. In this chapter, we dilate the social scale to address this pressing philosophical and social problem. (shrink)
Model Explanation Versus Model-Induced Explanation.Insa Lawler &Emily Sullivan -2021 -Foundations of Science 26 (4):1049-1074.detailsScientists appeal to models when explaining phenomena. Such explanations are often dubbed model explanations or model-based explanations. But what are the precise conditions for ME? Are ME special explanations? In our paper, we first rebut two definitions of ME and specify a more promising one. Based on this analysis, we single out a related conception that is concerned with explanations that are induced from working with a model. We call them ‘model-induced explanations’. Second, we study three paradigmatic cases of alleged (...) ME. We argue that all of them are MIE, upon closer examination. Third, we argue that this undermines the building consensus that model explanations are special explanations that, e.g., challenge the factivity of explanation. Instead, it suggests that what is special about models in science is the epistemology behind how models induce explanations. (shrink)
Vectors of epistemic insecurity.Emily Sullivan &Mark Alfano -2020 - In Ian James Kidd, Quassim Cassam & Heather Battaly,Vice Epistemology. New York, NY: Routledge.detailsEpistemologists have addressed a variety of modal epistemic standings, such as sensitivity, safety, risk, and epistemic virtue. These concepts mark out the ways that beliefs can fail to track the truth, articulate the conditions needed for knowledge, and indicate ways to become a better epistemic agent. However, it is our contention that current ways of carving up epistemic modality ignore the complexities that emerge when individuals are embedded within a community and listening to a variety of sources, some of whom (...) are intentionally engaged in deception or bullshit. In this context we want our beliefs to be secure. In this paper we translate the epistemic modal standing of safety into a framework appropriate for social epistemology and argue for the importance of epistemic network-security and belief-security to be added to this framework. We discuss the virtues that are salient for promoting network-security and the vices that undermine it. In particular, we highlight monitoring, adjusting, and restructuring virtues and vices. Importantly, each of these vices can be other-regarding or self-regarding. For example, one tempting way of dealing with insecurity within a network is to completely cut oneself off from biased sources. However, we argue that this is a self-regarding restructuring vice because it closes oneself off from opportunities for epistemic growth. By contrast, an other-regarding restructuring vice would be to cut off others from hearing from sources of information that would make their network more secure. (shrink)
How Values Shape the Machine Learning Opacity Problem.Emily Sullivan -2022 - In Insa Lawler, Kareem Khalifa & Elay Shech,Scientific Understanding and Representation: Modeling in the Physical Sciences. New York, NY: Routledge. pp. 306-322.detailsOne of the main worries with machine learning model opacity is that we cannot know enough about how the model works to fully understand the decisions they make. But how much is model opacity really a problem? This chapter argues that the problem of machine learning model opacity is entangled with non-epistemic values. The chapter considers three different stages of the machine learning modeling process that corresponds to understanding phenomena: (i) model acceptance and linking the model to the phenomenon, (ii) (...) explanation, and (iii) attributions of understanding. At each of these stages, non-epistemic values can, in part, determine how much machine learning model opacity poses a problem. (shrink)
Can Real Social Epistemic Networks Deliver the Wisdom of Crowds?Emily Sullivan,Max Sondag,Ignaz Rutter,Wouter Meulemans,Scott Cunningham,Bettina Speckmann &Mark Alfano -2014 - In Tania Lombrozo, Joshua Knobe & Shaun Nichols,Oxford Studies in Experimental Philosophy, Volume 1. Oxford, GB: Oxford University Press UK.detailsIn this paper, we explain and showcase the promising methodology of testimonial network analysis and visualization for experimental epistemology, arguing that it can be used to gain insights and answer philosophical questions in social epistemology. Our use case is the epistemic community that discusses vaccine safety primarily in English on Twitter. In two studies, we show, using both statistical analysis and exploratory data visualization, that there is almost no neutral or ambivalent discussion of vaccine safety on Twitter. Roughly half the (...) accounts engaging with this topic are pro-vaccine, while the other half are con-vaccine. We also show that these two camps rarely engage with one another, and that the con-vaccine camp has greater epistemic reach and receptivity than the pro-vaccine camp. In light of these findings, we question whether testimonial networks as they are currently constituted on popular fora such as Twitter are living up to their promise of delivering the wisdom of crowds. We conclude by pointing to directions for further research in digital social epistemology. (shrink)
From Explanation to Recommendation: Ethical Standards for Algorithmic Recourse.Emily Sullivan &Philippe Verreault-Julien -forthcoming -Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AIES’22).detailsPeople are increasingly subject to algorithmic decisions, and it is generally agreed that end-users should be provided an explanation or rationale for these decisions. There are different purposes that explanations can have, such as increasing user trust in the system or allowing users to contest the decision. One specific purpose that is gaining more traction is algorithmic recourse. We first pro- pose that recourse should be viewed as a recommendation problem, not an explanation problem. Then, we argue that the capability (...) approach provides plausible and fruitful ethical standards for re- course. We illustrate by considering the case of diversity constraints on algorithmic recourse. Finally, we discuss the significance and implications of adopting the capability approach for algorithmic recourse research. (shrink)
Negative epistemic exemplars.Mark Alfano &Emily Sullivan -2019 - In Benjamin R. Sherman & Stacey Goguen,Overcoming Epistemic Injustice: Social and Psychological Perspectives. London: Rowman & Littlefield International.detailsIn this chapter, we address the roles that exemplars might play in a comprehensive response to epistemic injustice. Fricker defines epistemic injustices as harms people suffer specifically in their capacity as (potential) knowers. We focus on testimonial epistemic injustice, which occurs when someone’s assertoric speech acts are systematically met with either too little or too much credence by a biased audience. Fricker recommends a virtuetheoretic response: people who do not suffer from biases should try to maintain their disposition towards naive (...) testimonial justice, and those who find themselves already biased should cultivate corrective testimonial justice by systematically adjusting their credence in testimony up or down depending on whether they are hearing from someone whom they may be biased against or in favor of. We doubt that the prominent admiration-emulation model of exemplarism will be much use in this connection, so we propose two ways of learning from negative exemplars to better conduct one’s epistemic affairs. In the admiration-emulation model, both the identification of what a virtue is and the cultivation of virtues identified thusly proceed through the admiration of virtuous exemplars. We show that this model has serious flaws and argue for two alternatives: the envy-agonism model and the ambivalence-avoidance model. (shrink)
Motivated numeracy and active reasoning in a Western European sample.Paul Connor,Emily Sullivan,Mark Alfano &Nava Tintarev -2020 -Behavioral Public Policy 1.detailsRecent work by Kahan et al. (2017) on the psychology of motivated numeracy in the context of intracultural disagreement suggests that people are less likely to employ their capabilities when the evidence runs contrary to their political ideology. This research has so far been carried out primarily in the USA regarding the liberal–conservative divide over gun control regulation. In this paper, we present the results of a modified replication that included an active reasoning intervention with Western European participants regarding both (...) the hierarchy–egalitarianism and individualism–collectivism divides over immigration policy (n = 746; considerably less than the preregistration sample size). We reproduce the motivated numeracy effect, though we do not find evidence of increased polarization of high-numeracy participants. (shrink)
The wisdom-of-crowds: an efficient, philosophically-validated, social epistemological network profiling toolkit.Colin Klein,Marc Cheong,Marinus Ferreira,Emily Sullivan &Mark Alfano -2023 - In Hocine Cherifi, Rosario Nunzio Mantegna, Luis M. Rocha, Chantal Cherifi & Salvatore Miccichè,Complex Networks and Their Applications XI: Proceedings of The Eleventh International Conference on Complex Networks and Their Applications: COMPLEX NETWORKS 2022 — Volume 1. Springer.detailsThe epistemic position of an agent often depends on their position in a larger network of other agents who provide them with information. In general, agents are better off if they have diverse and independent sources. Sullivan et al. [19] developed a method for quantitatively characterizing the epistemic position of individuals in a network that takes into account both diversity and independence; and presented a proof-of-concept, closed-source implementation on a small graph derived from Twitter data [19]. This paper reports on (...) an open-source reimplementation of their algorithm in Python, optimized to be usable on much larger networks. In addition to the algorithm and package, we also show the ability to scale up our package to large synthetic social network graph profiling, and finally demonstrate its utility in analyzing real-world empirical evidence of ‘echo chambers’ on online social media, as well as evidence of interdisciplinary diversity in an academic communications network. (shrink)
A normative framework for sharing information online.Emily Sullivan &Mark Alfano -2021 - In Carissa Véliz,The Oxford Handbook of Digital Ethics. Oxford University Press.detailsPeople have always shared information through chains and networks of testimony. It’s arguably part of what makes us human and enables us to live in cooperative communities with populations greater than the Dunbar number. The invention of the Internet and the rise of social media have turbo-charged our ability to share information. In this chapter, we develop a normative framework for sharing information online. This framework takes into account both ethical and epistemic considerations that are intertwined in typical cases of (...) online testimony. We argue that, while the current state of affairs is not entirely novel, recent technological developments call for a rethinking of the norms of testimony, as well as the articulation of a set of virtuous dispositions that people would do well to cultivate in the capacity as conduits (not just sources or receivers) of information. (shrink)
Ethical pitfalls for natural language processing in psychology.Mark Alfano,Emily Sullivan &Amir Ebrahimi Fard -forthcoming - In Morteza Dehghani & Ryan Boyd,The Atlas of Language Analysis in Psychology. Guilford Press.detailsKnowledge is power. Knowledge about human psychology is increasingly being produced using natural language processing (NLP) and related techniques. The power that accompanies and harnesses this knowledge should be subject to ethical controls and oversight. In this chapter, we address the ethical pitfalls that are likely to be encountered in the context of such research. These pitfalls occur at various stages of the NLP pipeline, including data acquisition, enrichment, analysis, storage, and sharing. We also address secondary uses of the results (...) and tools developed through psychometric NLP, such as profit-driven targeted advertising, political campaigns, and domestic and international psyops. Along the way, we reflect on potential ethical guidelines and considerations that may help researchers navigate these pitfalls. (shrink)
Big Data Idealizations.Emily Sullivan -unknowndetailsTalk at the Philosophy [in:of:for:and] Digital Knowledge Infrastructures online workshop (08/09/2022).
No categories
The Free Market and the Human Condition: Essays on Economics and Culture.Jeremy Beer,Bryce Christensen,Kirk Fitzpatrick,Pamela Hood,William H. Krieger,Peter McNamara,Emily Sullivan &Lee Trepanier (eds.) -2014 - Lexington Books.detailsThe Free Market and the Human Condition explores the human condition as situated in the free market from a variety of academic disciplines. By relying upon contributors who approach the topic from their respective disciplines, the book provides an accumulated picture of the free market, the human condition, and the relationship between them.
Understanding the Virtue-Relevant Self Through Courage.Cynthia Pury,Charles Starkey &Emily Sullivan -unknowndetailsTo what extent do differences in who we are predict differences in courage? We propose to de-velop a measure of the virtue-relevant self, which is composed of self-conception, social roles, virtue-relevant values, and personality traits. We will then conduct three studies using this meas-ure to determine the extent to which these various components of the virtue-relevant self pre-dict the types of acts people consider courageous as well as the willingness of people to engage in courageous acts themselves. We believe that (...) individual differences in each of these compo-nents – that is, the content of the virtue-relevant self – will correlate with differences in first, how people rate actions that they themselves have undertaken in the past; second, how people rate actions that other people have taken; and third, the willingness of people to take certain kinds of courageous action. If found, these relations will have broader implications for the self and virtues by indicating that traits of the self beyond character traits affect both the conception of virtuous behavior and virtuous behavior itself. (shrink)