This talk page is intended for discussion of template content and links. For discussions relating to technical aspects of the Find sources template, seeTemplate talk:Find sources
This module is part ofWikiProject Reliability, a collaborative effort to improve thereliability of Wikipedia articles. If you would like to participate, please visit theproject page, where you can join thediscussion and see alist of open tasks.ReliabilityWikipedia:WikiProject ReliabilityTemplate:WikiProject ReliabilityReliability
Can the logic here convert "List of foo" as a parameter to become a search for "foo"? e.g. "List of fictional rutabagas" becomes "fictional rutabagas" for search purposes? This would help correctly identify relevant sources in AfDs, and those who actually meant to search for "List of foo" can add the prefix back if desired. My experience is that "list of foo" consistently fails to produce any relevant sources, while "foo" will produce more sources, with some arguably relevant to the discussion.Jclemens (talk)05:59, 20 August 2024 (UTC)[reply]
Awesome Aasim, Can you please elaborate on your intentions withthis sandbox edit ? I believe that it would be a perversion of this module and the associated template to admit any notion of AI bots into the module configuration either as 1) a reliable source, or as 2) a good way to find reliable sources (their hallucinations are legion). Hence, I would be against porting your changes to the module or to the template without consensus achieved at an Rfc on this page advertised at the main venues where AI bots are being discussed, as well as atWP:VPR. Thanks,Mathglot (talk)04:51, 16 October 2024 (UTC)[reply]
I kind of agree as well. However, Google and Bing also index unreliable sources. I was initially going to put in a query "find reliable sources for $1" but then decided against it for some reason.AwesomeAasim14:00, 16 October 2024 (UTC)[reply]
That is, of course, true. However, I have never seen Google or Bing invent a very convincing-looking source out of wholecloth bits, creating authors who do not exist, or very complex and scholarly-looking titles that look real but do not exist (but are quite similar to pieces of other titles that do exist), and so on. If the task assigned were to add sources to an article that do not exist but would rarely get challenged, AI bots are definitely the way to go. It's quite possible vandals or lazy or clueless editors are doing this already, and it is a problem that will have to be addressed at some point.Mathglot (talk)15:34, 16 October 2024 (UTC)[reply]
When a chatbot is powered by a search engine, it is less likely to make stuff up; but that doesn't mean it doesn't pull from unreliable sources. I have tried Copilot before (not necessarily for Wikipedia tasks, but for personal tasks like clarifying math concepts) and it has not really failed me. On the other hand, ChatGPT has occasionally made stuff up, especially when it does not query from the web.AwesomeAasim15:45, 16 October 2024 (UTC)[reply]
Imho, this page is not the place for Wikipedia editors to debate whether chatbots are more or less likely to hallucinate under this or that circumstance, and, pardon me, but anecdotal evidence about failure to fail in casual use by non-experts is close to worthless. Please use the AI discussion venues for that. Here we should debate whether a find sources module should use the results of AI, however triggered, and imho the answer to that is a slam-dunk 'no'. I will shut up now, and hopefully others will chime in.Mathglot (talk)16:26, 16 October 2024 (UTC)[reply]
Google's top-level "AI overview" results are rarely totally accurate, but they do give the source links, which in aggregate ends up being significantlymore reliable for us than their raw top search results, which, often being shit like quora or low-quality zines, do not provide sources at all.
I don't know if linking to another engine prompting something like ChatGPT would get better AI-enhanced results than Google or Bing in that respect, since they are putting a good deal of effort into making it give back real online-accessible sources. That seems to align with our goal here.
Of course the other question is whether it's more enticing for the novice editor to have a shiny link saying "ChatGPT" or whatever latest AI tool is out there, instead of just clicking "Google", even if the result is the same. Also, calling exclusively the Google AI overview results does not appear possible right now as it's still considered an experimental feature, and it doesn't appear to be available in private browsing either. But that's my thought going forward.SamuelRiv (talk)17:52, 16 October 2024 (UTC)[reply]
I think that even if chatbots were 100% accurate in their output, we would still want to avoid using them for use-cases like this because of how heavily these technologies rely on Wikipedia itself as an information source. We need to stay upstream of LLMs to avoid circular referencing.signed,Rosguilltalk16:47, 16 October 2024 (UTC)[reply]
I wonder if with the right prompt engineering we can get these chatbots to actually spit out reliable sources. We can maybe base our entire prompt based on something likeWP:RSPS. Although it probably would overfill the query parameter.AwesomeAasim17:25, 16 October 2024 (UTC)[reply]
Are you proposing to engineer a solution that operates without human intervention? Because if you are successful, you should quit your day job and launch the next AI start-up, or become CTO of one of the existing ones.Mathglot (talk)17:30, 16 October 2024 (UTC)[reply]
No, a person will still need to click on and then review what is provided, and provide follow up queries. The queries I pass into the ?q=... parameter may be a good start, but I don't think they are a good end. But us Wikipedians should know this; just as the first page of Google/Bing search results can at times be littered with stuff likeWP:DAILYFAIL andWP:NEWSMAX, depending on the query and depending on previous searches.
I have largely toyed with ChatGPT and found it is not always good. Web-based LLMs like Copilot and Gemini are a bit better, although I remember they had a bumpy start, sometimes pulling nonsense from places like Reddit and Facebook.
I do not agree that we should just copy and paste the exact output of an LLM. I only think AI is good to assist humans, but practically can never replace humans.AwesomeAasim17:38, 16 October 2024 (UTC)[reply]
Done by Sdkb inSpecial:Diff/1195169672SWinxy (talk)22:16, 12 January 2024 (UTC)[reply]