Movatterモバイル変換

[0]ホーム
URL:

Keyboard Shortcuts

Thread View

  • j: Next unread message
  • k: Previous unread message
  • j a: Jump to all threads
  • j l: Jump to MailingList overview
List overview
Download

Wikitech-lMay 2019

wikitech-l@lists.wikimedia.org
  • 51 participants
  • 59 discussions
Start a nNew thread
Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25
Until some weeks agohttp://dumps.wikimedia.org/backup-index.html usedto show 4 dumps in progress at the same time. That meant that newdatabase dumps normally was available within about 3 weeks for alldatabases except for enwiki and maybe dewiki where the dump process dueto size took longer time.However the 4 dumps processes at one time become 3 some weeks ago. Andafter massive failures at June 4, only one dump has been in progress atthe same time. So at the current speed it will take several months tocome thru all dumps.Is it possible to speed up the process again using several dumpprocesses at the same time?Thank you,Byrial
3 2
0 0
User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25
Hi!from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites.Domas
19 61
0 0

17 Apr '25
Hoi,This is an inquiry from my friend in academia, researching about Wikipedia.He would like to know whether there's a way to acquire a list of templatesincluding external links. Here are some examples including external links.https://ja.wikipedia.org/wiki/Template:JOI/dochttps://ja.wikipedia.org/wiki/Template:Twitter/docSuch links are stored in externallinks.sql.gz, in an expanded form.When you want to check increase/decrease of linked domains in chronologicalorder through edit history, you have to check pages-meta-history1.xml etc.In a such case, traditional links and links by templates are mixed,Therefore, the latter ones (links by templates) should be expanded totraditional link forms.Sorry if what I am saying does not make sense.Thanks in advance,--Takashi Ota [[U:Takot]]
13 24
0 0
EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25
What's the status of the project to create a grammar for Wikitext in EBNF?There are two pages:http://meta.wikimedia.org/wiki/Wikitext_Metasyntaxhttp://www.mediawiki.org/wiki/Markup_specNothing seems to have happened since January this year. Also the comments onthe latter page seem to indicate a lack of clear goal: is this just a funproject, is it to improve the existing parser, or is it to facilititate anew parser? It's obviously a lot of work, so it needs to be of clearbenefit.Brion requested the grammar IIRC (and there's a comment to that effect athttp://bugzilla.wikimedia.org/show_bug.cgi?id=7), so I'm wondering what became of it.Is there still a goal of replacing the parser? Or is there some alternativeplan?Steve
26 217
0 0
Missing Section Headings
by Marc Riddell 13 Sep '24

13 Sep '24
Hello,I have been a WP editor since 2006. I hope you can help me. For some reasonI no longer have Section Heading titles showing in the Articles. This istrue of all Headings including the one that carries the Article subject'sname. When there is a Table of Contents, it appears fine and, when I clickon a particular Section, it goes to that Section, but all that is there is astraight line separating the Sections. There is also no button to edit aSection. If I edit the page and remove the "== ==" markers from the SectionTitles, the Title then shows up, but not as a Section Heading. Also, I don'thave any Date separators on my Want List. This started 2 days ago. Anythoughts?Thanks,Marc Riddell[[User:Michael David]]
10 11
0 0
I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here.tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.)(Is there a place onmw.org to put this tidbit on?)------- Forwarded message -------From: "Brian Levine" <support(a)github.com> (GitHub Staff)To: matma.rex(a)gmail.comCc:Subject: Re: Commits in mirrored repositories not showing up on my profileDate: Tue, 09 Jul 2013 06:47:19 +0200Hi BartoszIn order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit.The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits.CheersBrian-- Matma Rex
3 3
0 0
Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24
We just released a new version of Research:FAQ on Meta [1], significantlyexpanded and updated, to make our processes at WMF more transparent and tomeet an explicit FDC request to clarify the role and responsibilities ofindividual teams involved in research across the organization.The previous version – written from the perspective of the (now inactive)Research:Committee, and mostly obsolete since the release of WMF's openaccess policy [2] – can still be found here [3].Comments and bold edits to the new version of the document are welcome. Forany question or concern, you can drop me a line or ping my username on-wiki.Thanks,Dario[1]https://meta.wikimedia.org/wiki/Research:FAQ[2]https://wikimediafoundation.org/wiki/Open_access_policy[3]https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953*Dario Taraborelli *Head of Research, Wikimedia Foundationwikimediafoundation.org •nitens.org • @readermeter<http://twitter.com/readermeter>
2 1
0 0
bluejeans
by Jeremy Baron 22 Feb '23

22 Feb '23
Hi,On Tue, Mar 1, 2016 at 3:36 PM, David Strine <dstrine(a)wikimedia.org> wrote:> We will be holding this brownbag in 25 minutes. The Bluejeans link has> changed:>>https://bluejeans.com/396234560I'm not familiar with bluejeans and maybe have missed a transitionbecause I wasn't paying enough attention. is this some kind ofexperiment? have all meetings transitioned to this service?anyway, my immediate question at the moment is how do you join withoutsharing your microphone and camera?am I correct thinking that this is an entirely proprietary stackthat's neither gratis nor libre and has no on-premise (not cloud)hosting option? are we paying for this?-Jeremy
9 16
0 0

04 Jul '20
As of 950cf6016c, the mediawiki/core repo was updated to use DB_REPLICAinstead of DB_SLAVE, with the old constant left as an alias. This is partof a string of commits that cleaned up the mixed use of "replica" and"slave" by sticking to the former. Extensions have not been massconverted. Please use the new constant in any new code.The word "replica" is a bit more indicative of a broader range of DBsetups*, is used by a range of large companies**, and is more neutral inconnotations.Drupal and Django made similar updates (even replacing the word "master"):*https://www.drupal.org/node/2275877*https://github.com/django/django/pull/2692/files &https://github.com/django/django/commit/beec05686ccc3bee8461f9a5a02c607a023…I don't plan on doing anything to DB_MASTER, since it seems fine by itself,like "master copy", "master tape" or "master key". This is analogous to amaster RDBMs database. Even multi-master RDBMs systems tend to have astronger consistency than classic RDBMs slave servers, and presentthemselves as one logical "master" or "authoritative" copy. Even in it'spersonified form, a "master" database can readily be thought of asanalogous to "controller", "governer", "ruler", lead "officer", or such.*** clusters using two-phase commit, galera using certification-basedreplication, multi-master circular replication, ect...**https://en.wikipedia.org/wiki/Master/slave_(technology)#Appropriateness_of_…***http://www.merriam-webster.com/dictionary/master?utm_campaign=sd&utm_medium…-- -Aaron
4 3
0 0
Finding and fixing cyclic dependencies
by Daniel Kinzler 25 Jun '19

25 Jun '19
Hi all!I invite you to try out "Project Ruprecht"[1][2], a tool that measures the"tangledness" of PHP code, and provides you with a "naughty list" of things to fix.For now, you will have to install this locally. I hope however to soon have thisrun automatically against core on a regular basis, perhaps by integrating itwith SonarQube. Maybe some day we can also integrate it with CI, to generate awarning when a new cyclic dependency is about to be introduced.So that was the tl;dr. Now for some context, history, and shout-outs. And someactual real world science, too!For a while now, I have been talking about the how much of a problem cyclicdependencies are in MediaWiki core: When two components (classes, namespaces,libraries, whatever) depend on each other, directly or indirectly, this meansthat one cannot be used without the other, nor can it be tested, understood, ormodified without also considering the other. So, in effect, they behave as *one*component, not two. Applied to MW core, this means that roughly half of our 1600classes effectively behave like a single giant class. This makes the code ratherhard to deal with.To fix this, I have been looking for tools that let me identify "tangles" ofclasses that depend on each other, and metrics' to measure the progress of"untangling" the code. However, the classic code quality metrics focus on"local" properties of the code, so they can't tell us much about the progress ofuntangling. And the tools I found that would detect cyclic dependencies in PHPcode would all choke on MediaWiki core: they would try to list all detectedcycles - which, by the super-exponential nature of possible paths through agraph, would be millions and millions. So, the tools would choke and die. Thatapproach isn't practical for us.Two discoveries allowed me to come up with a working solution:First, I decided to leave the PHP world and turned towards graph analysis toolsbuilt for large data sets. Python's graph-tool did the trick. It's build on topof boost and numpy, and it's *fast*. It crunched through the 7500 or so classdependencies in MW core in a split second, and told me that we have 14 "tangles"(non-trivial strongly connected components), and that 43% of our classes are inthese tangles, with 40% being part of one big tangle that is essentially ourmonolith manifest. So now I had a metric to work with: the number of classes intangles.That was great, but still didn't tell me where to start. Graph-tool was stillnot fast enough to deal with millions of cycles, and even if it had been, thatdata wouldn't be very useful. I needed some smart heuristics. Luckily, I(totally unintentionally, promise!) nerd sniped[5] Amir Sarabadani one eveningat the WMDE office by telling him about this problem. The next day, he told methat he had been digging into the problem all night, and he had found a paperthat sounded relevant, and it also came with working code: "Breaking Cycles inNoisy Hierarchies"[3] by J. Sun, D. Ajwani, P.K. Nicholson, A. Sala, and S.Parthasarathy. I played with the code a bit, and yes! It spat out a list of 290or so dependencies[4] that it thought were bad - and I agree for a good numberof them. It's not a clean working list, but it gives a very good idea of whereto start looking.I find it quite fascinating that this works so well for cleaning up a codebase.After all, the heuristic wasn't design for this - it was designed for fixingmessy ontologies. Indeed, one of their test data sets was (English language)Wikipedia's category system! I'd love to see what it does with Wikidata'ssubclass hierarchy :)But I suppose it makes sense - dependencies in software are conceptually a lotlike an ontology, and the same strategies of stratification and abstractionapply. And the same difficulties, too - it's easy enough to spot a problematiccycle, but often hard to say where it should be cut. And how to cut it - often,the solution is not to just remove the dependency, but to introduce a newabstraction that allows the relationship to exist without a cycle. I'd love tosee the research continue in that direction!So, a big shout out to the researchers, and to Amir who found the paper!I hope my ramblings have made you curious to play with Ruprecht, and see what ithas to say about other code bases. There's also another feature to play withwhich I haven't discussed here: detection of risky classes using the Page Rankalgorithm. Fun!Cheers,Daniel[1]https://phabricator.wikimedia.org/diffusion/MTDA/repository/master/[2]https://gerrit.wikimedia.org/r/admin/projects/mediawiki/tools/dependency-an…[3]https://github.com/zhenv5/breaking_cycles_in_noisy_hierarchies[4]https://phabricator.wikimedia.org/P8513[5]https://xkcd.com/356/-- Daniel KinzlerPrincipal Software Engineer, Core PlatformWikimedia Foundation
3 5
0 0
Results per page:
[8]ページ先頭
©2009-2025 Movatter.jp