Movatterモバイル変換

[0]ホーム
URL:

Keyboard Shortcuts

Thread View

  • j: Next unread message
  • k: Previous unread message
  • j a: Jump to all threads
  • j l: Jump to MailingList overview
List overview
Download

Wikitech-lMarch 2025

wikitech-l@lists.wikimedia.org
  • 34 participants
  • 40 discussions
Start a nNew thread
Hi all,we're movingirc.wikimedia.org to a new infrastructure [1]. Thisservice is unrelated to the IRC network (Libera) we use for real timediscussions. Insteadirc.wikimedia.org is an IRC service forbroadcasting recent changes events from public Wikimedia wikis to beused by various bots connected to per-wiki IRC channels.irc.wikimedia.org should not be used for any new bots (which shouldrather use Eventstreams[2]), but we still have various important botsrelying on the legacy IRC-based infrastructure.The current setup is full of technical debt and ultimately based on apatched version of a very old release of ircd-ratbox with a relayservice written in Python 2.We are replacing it with a modern standalone implementation in Python,which broadcasts IRC notifications in a format compatible to what iscurrently in use by the legacy setup:https://github.com/paravoid/ircstreamLast week during the SRE Infrastructure Foundations hackathon theircstream production setup was created. All our tests have beensuccessful, so on Thursday October 10 at 08:00 UTC we'll switch theirc.wikimedia.org DNS name to the new setup.No changes are needed to any bots, but if you run into any issuesafter the switch, please notify us in the #wikimedia-sre-foundationsIRC channel or leave a note athttps://phabricator.wikimedia.org/T376014.Cheers,Luca, Simon and MoritzFootnotes:[1]https://wikitech.wikimedia.org/wiki/Irc.wikimedia.org[2]https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams_HTTP_Service
4 3
0 0
Greetings-With the security/maintenance release of MediaWiki 1.39.11/1.41.5/1.42.4,we would also like to provide this supplementary announcement of MediaWikiextensions and skins with now-public Phabricator tasks, security patchesand backports [1]:SocialProfile+ (T373265 <https://phabricator.wikimedia.org/T373265>, CVE-2025-23074) -Special:EditProfile exposes the contents of profile fields marked"hidden"/friends or "friends of friends" when the privileged user isn't afriend of the user whose profile they edit(ed)https://gerrit.wikimedia.org/r/q/I4b77ced314bc6cea0ef3657a82e7467d3661fe2aGlobalBlocking+ (T377855 <https://phabricator.wikimedia.org/T377855>, CVE-2025-23073) -API list=globalblocks can reveal IP of autoblock if username and IP areincluded in the bgtargets parameterhttps://gerrit.wikimedia.org/r/q/I2a2d32aedf6328be0a9f1b4e04a6567a25f19486RefreshSpecial+ (T378885 <https://phabricator.wikimedia.org/T378885>, CVE-2025-23072) -XSS in Special:RefreshSpecialhttps://gerrit.wikimedia.org/r/q/Ic9547e80a8296d707ad8a157eb8ba7aa26fb08dcDataTransfer+ (T379749 <https://phabricator.wikimedia.org/T379749>, CVE-2025-23081) -Various security vulnerabilities in Extension:DataTransferhttps://gerrit.wikimedia.org/r/q/I773c616db781d2f3f30893ad01ef503bf251a2b3https://gerrit.wikimedia.org/r/q/I7c9de4c8dcdb3276ba923c6bc7c8eef3531324c7https://gerrit.wikimedia.org/r/q/I9223c31f02f31f1e06e1a8cddf7d539cc8d3a3d9https://gerrit.wikimedia.org/r/q/I5e1538a3bf66378810f905834c05626e1d2c82f0https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DataTransfer/+/1093931https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DataTransfer/+/1080451OpenBadges+ (T381220 <https://phabricator.wikimedia.org/T381220>, CVE-2025-23080) -XSSes in Special:BadgeViewhttps://gerrit.wikimedia.org/r/q/Ic9448312fa7f1cbc8feac3f852bc8720568522e2ArticleFeedbackv5+ (T381753 <https://phabricator.wikimedia.org/T381753>, CVE-2025-23079) -XSSes in Extension:ArticleFeedbackv5https://gerrit.wikimedia.org/r/q/I6ee51c8b518bda41739fd666fa2891cc12e79ac3BreadCrumbs2+ (T382043 <https://phabricator.wikimedia.org/T382043>, CVE-2025-23078) -XSS in BreadCrumbs2https://gerrit.wikimedia.org/r/q/I7878f8f7bc067080f80427b90f8d85337f172711The Wikimedia Security Team recommends updating these extensions and/orskins to the current master branch or relevant, supported release branch[2] as soon as possible. Some of the referenced Phabricator tasks above_may_ still be private. Unfortunately, when security issues are reported,sometimes sensitive information is exposed and since Phabricator ishistorical, we cannot make these tasks public without exposing thissensitive information. If you have any additional questions or concernsregarding this update, please feel free to contact security(a)wikimedia.orgor file a security task within Phabricator [3].[1]https://phabricator.wikimedia.org/T375631[2]https://www.mediawiki.org/wiki/Version_lifecycle[3]https://www.mediawiki.org/wiki/Reporting_security_bugs
1 0
0 0

23 Jul '25
Hi all,With MediaWiki at the WMF moving to Kubernetes, it's now time to startrunning manual maintenance scripts there. Any time you would previously SSHto a mwmaint host and run mwscript, follow these steps instead. The old waywill continue working for a little while, but it will be going away.What's familiar:Starting a maintenance script looks like this: rzl@deploy2002:~$ mwscript-k8s --comment="T341553" -- Version.php--wiki=enwikiAny options for the mwscript-k8s tool, as described below, go before the --.After the --, the first argument is the script name; everything else ispassed to the script. This is the same as you're used to passing tomwscript.What's different:- Run mwscript-k8s on a deployment host, not the maintenance host. Eitherdeployment host will work; your job will automatically run in whicheverdata center is active, so you no longer need to change hosts when there’s aswitchover.- You don't need a tmux. By default the tool launches your maintenancescript and exits immediately, without waiting for your job to finish. Ifyou log out of the deployment host, your job keeps running on theKubernetes cluster.- Kubernetes saves the maintenance script's output for seven days aftercompletion. By default, mwscript-k8s prints a kubectl command that you (oranyone else) can paste and run to monitor the output or save it to a file.- As a convenience, you can pass -f (--follow) to mwscript-k8s to immediatelybegin tailing the script output. If you like, you can do this inside a tmuxand keep the same workflow as before. Either way, you can safely disconnectand your script will continue running on Kubernetes. rzl@deploy2002:~$ mwscript-k8s -f -- Version.php --wiki=testwiki [...] MediaWiki version: 1.43.0-wmf.24 LTS (built: 22:35, 23 September 2024)- For scripts that take input on stdin, you can pass --attach tomwscript-k8s, either interactively or in a pipeline. rzl@deploy2002:~$ mwscript-k8s --attach -- shell.php --wiki=testwiki [...] Psy Shell v0.12.3 (PHP 7.4.33 — cli) by Justin Hileman > $wmgRealm = "production" > rzl@deploy2002:~$ cat example_url.txt | mwscript-k8s --attach --purgeList.php [...] Purging 1 urls Done!- Your maintenance script runs in a Docker container which will not outliveit, so it can't save persistent files to disk. Ensure your script logs itsimportant output to stdout, or persists it in a database or other remotestorage.- The --comment flag sets an optional (but encouraged) descriptive label,such as a task number.- Using standard kubectl commands[1][2], you can check the status, and viewthe output, of your running jobs or anyone else's. (Example: `kube_envmw-script codfw; kubectl get pod -l username=rzl`)[1]:https://wikitech.wikimedia.org/wiki/Kubernetes/Kubectl[2]:https://kubernetes.io/docs/reference/kubectl/quick-reference/What's not supported yet:- Maintenance scripts launched automatically on a timer. We're working onmigrating them -- for now, this is for one-off scripts launched by hand.- If your job is interrupted (e.g. by hardware problems), Kubernetes canautomatically move it to another machine and restart it, babysitting ituntil it completes. But we only want to do that if your job is safe torestart. So by default, if your job is interrupted, it will stay stoppeduntil you restart it yourself. Soon, we'll add an option to declare "thisis idempotent, please restart it as needed" and that design is recommendedfor new scripts.- No support yet for mwscriptwikiset, foreachwiki, foreachwikiindblist,etc, but we'll add similar functionality as flags to mwscript_k8s.Your feedback:Let me know by email or IRC, or on Phab (T341553<https://phabricator.wikimedia.org/T341553>). If mwscript-k8s doesn't workfor you, for now you can fall back to using the mwmaint hosts as before --but they will be going away. Please report any problems sooner rather thanlater, so that we can ensure the new system meets your needs before thathappens.Thanks,Reuven, for Service Ops SRE
9 23
0 0

24 Apr '25
Hello, in the next coming months, these changes will happen in databasesand the infrastructure. And it might affect you if you rely on them in yourtools or queries. This list is ordered based on how soon the change willhappen.We understand that updating your tools and systems can be time consuming,hence we are giving an advanced notice. I truly apologize for theinconvenience but many of these changes are needed to keep the site runningsmoothly.Image table redesignAround fourteen years after the creation of T28741<https://phabricator.wikimedia.org/T28741>, we are implementing the changesdescribed therein. Currently, every current version of an image has a rowin the image table and if there are older versions of that file, those rowscould be found in the oldimage table. These two tables (image and oldimage)will be dropped in around two months. The replacement will be two maintables: file and filerevision. Every file will have a row in the file tabledescribing the name and the type. Every version of the file (current andold) will have a row in filerevision describing the file-specificinformation such as its size or the hash of the file, similar to theexisting distinction between pages and revisions. Another improvement isthat every file and file revision will get a unique auto increment idsimplifying many operations and queries. You can check T28741<https://phabricator.wikimedia.org/T28741> for more information. The newtables are already accessible in wikireplicas but the data hasn’t beenfully migrated yet.Term store split out of wikidata’s databaseWikidata’s database has been growing too fast and we need to move the termstore (tables starting with wbt_) to a dedicated cluster to allow growthand improve wikidata’s performance by utilizing cache locality. The newsection will be called x3 and you will be able to access it in wikireplicasbut this also means you won’t be able to join these tables with the rest ofwikidata’s database (such as page table) since they will be residing in twophysically separate servers that also means most of your queries towikidata’s database (and term store) will become faster. We are aiming forthe switch to happen in three months’ time. You can follow the work inT351820 <https://phabricator.wikimedia.org/T351820>.Additionally, wb_type table will be dropped and the mapping will behard-coded in the code instead. See gerrit:1110810<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110810>for more details. This helped us simplify a lot of Wikibase code (example<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110720>).Categorylinks normalizationCategorylinks is the next table in the series of links tables beingnormalized via the linktarget table (parent ticket<https://phabricator.wikimedia.org/T300222>, RFC<https://phabricator.wikimedia.org/T222224>). Similar to templatelinks andpagelinks tables, cl_to will be dropped and instead the new fieldcl_target_id will point to lt_id in the linktarget table. We will also dropthe cl_collation field and replace it with cl_collation_id which will pointto the collation_id field on the new table we are introducing calledcollation. We are aiming to get this fully done by the end of the nextquarter (end of June 2025) but it depends on how fast the migration scriptcan operate and that’s outside of our control. You can follow the work inT299951 <https://phabricator.wikimedia.org/T299951>.It’s worth noting thatafter this migration is done, we will start working on the imagelinks table.Thank you-- *Amir Sarabadani (he/him)*Staff Database ArchitectWikimedia Foundation <https://wikimediafoundation.org/>
1 1
0 0

17 Apr '25
Hi Sir/Madam,I hope you’re doing well.I am Nidha Ahmed, and a junior undergraduate student studying ArtificialIntelligence and Data Science. I’m also going to join Google as an SWEIntern next summer.I’ve always been a huge admirer of the Wikimedia Foundation and its missionto make free educational content accessible to everyone. It’s incrediblehow your work empowers millions of learners around the globe, and I’m veryeager to be part of this impact.I have experience with JavaScript, HTML, CSS, jQuery, ReactJS, and Python,and I’ve dabbled a bit in Android development as well. While I’m new to PHPand Ruby on Rails, I’m super excited to learn them! If you could share someresources or tips to get started with these technologies, it would be ahuge help.A little about me: I’ve worked on projects like a movie recommendationsystem, a text summarization web app, and a geospatial analysis tool forpublic health. I’ve also contributed to open-source initiatives likedesigning real-time heatmaps for SoundScape. I’m looking to grow my skillset, hence I’m so drawn to Wikimedia.It would be helpful for me to get guidance on how to start contributing.I’m genuinely excited about the chance to learn from your community andsupport your amazing mission.I'm attaching my resume below for reference and proofs.Looking forward to hearing from you!Best regards,Nidha Ahmed Mohammad.
3 2
0 0
Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25
Until some weeks agohttp://dumps.wikimedia.org/backup-index.html usedto show 4 dumps in progress at the same time. That meant that newdatabase dumps normally was available within about 3 weeks for alldatabases except for enwiki and maybe dewiki where the dump process dueto size took longer time.However the 4 dumps processes at one time become 3 some weeks ago. Andafter massive failures at June 4, only one dump has been in progress atthe same time. So at the current speed it will take several months tocome thru all dumps.Is it possible to speed up the process again using several dumpprocesses at the same time?Thank you,Byrial
3 2
0 0
User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25
Hi!from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites.Domas
19 61
0 0
API issue
by ovskmendov＠proton.me 17 Apr '25

17 Apr '25
Whenever I access the API, I get "Wikimedia Error: Our servers are currently under maintenance or experiencing a technical problem." Sometimes if I wait a bit, I'll get a few good requests, but it will shortly go back to the previous message. My code was working just fine a few days ago and I didn't change anything.Sent with [Proton Mail](https://proton.me/) secure email.
7 12
0 0

17 Apr '25
Hoi,This is an inquiry from my friend in academia, researching about Wikipedia.He would like to know whether there's a way to acquire a list of templatesincluding external links. Here are some examples including external links.https://ja.wikipedia.org/wiki/Template:JOI/dochttps://ja.wikipedia.org/wiki/Template:Twitter/docSuch links are stored in externallinks.sql.gz, in an expanded form.When you want to check increase/decrease of linked domains in chronologicalorder through edit history, you have to check pages-meta-history1.xml etc.In a such case, traditional links and links by templates are mixed,Therefore, the latter ones (links by templates) should be expanded totraditional link forms.Sorry if what I am saying does not make sense.Thanks in advance,--Takashi Ota [[U:Takot]]
13 24
0 0

02 Apr '25
Hello all!The Search Platform Team usually holds an open meeting on the firstWednesday of each month. Come talk to us about anything related toWikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons QueryService (WCQS), etc.!Feel free to add your items to the Etherpad Agenda for the next meeting.Details for our next meeting: Date: Wednesday, April 2, 2025 Time: 15:00-16:00 UTC / 08:00 PST / 11:00 EST / 17:00 CET Etherpad:https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link:https://meet.google.com/vgj-bbeb-uyi Join by phone:https://tel.meet/vgj-bbeb-uyi?pin=8118110806927Have fun and see you soon! Guillaume-- *Guillaume Lederrey* (he/him)Engineering ManagerWikimedia Foundation <https://wikimediafoundation.org/>
1 1
0 0
Results per page:
[8]ページ先頭
©2009-2025 Movatter.jp