Movatterモバイル変換

[0]ホーム
URL:

Keyboard Shortcuts

Thread View

  • j: Next unread message
  • k: Previous unread message
  • j a: Jump to all threads
  • j l: Jump to MailingList overview
List overview
Download

Wikitech-lMay 2002

wikitech-l@lists.wikimedia.org
  • 22 participants
  • 89 discussions
Start a nNew thread
response times
by Lars Aronsson 22 Aug '02

22 Aug '02
Today Friday, the front page of the English Wikipedia has been fastall day.Another page (I monitorhttp://www.wikipedia.com/wiki/Sweden) was slowfor one period of 30 minutes (09:30-10:00 am GMT) and another periodof two hours (11:40-13:50 GMT). Some other URLs on the internationalWikipedias were also affected at the same time. This might be due tomaintenance or work being done on the scripts.Subtract 7 hours from GMT to get the server's local time zone(PDT = GMT -0700).Apart from these two limited intervals, every URL that I monitor havebeen fast all day, including the recent changes pages.I'm very happy with this, and hope Brion and Jimmy (and who else?)will soon get the talk namespace links back without hurtingperformance. (But hey, never make big fixes five minutes before youleave for the weekend! Better just leave it as is if you have to go.)And now for some more relaxed Friday reading, actually related toperformance problems. (The following analysis might be politicallyslanted. Don't take it too seriously.) The Swedish parliamentelections are coming up in September, so the political parties arestarting up their campaigns. The problem is there are no big issuesto fight about. The four non-socialist parties have unusually boringcandidates (Dukakis style), and everybody expects the currentsocial-democratic government to win. The single issue that seems tobe coming up is the national sick leave insurance, which is paid bytax money, and far over budget. This is linked to the fact that"burn-out" is now an accepted medical diagnosis for which you areallowed to take a long sick leave on the tax payers' expense. Youwould expect such welfare excesses to be on the social democratagenda, and that non-socialists would urge for tax cuts and a balancedbudget. However, the current s-d govt has been doing a great jobbalancing the budget, and they will now have to deal with cutting backthis overgenerous sick leave compensation without hurting theirvoters' feelings. Tough job. The Christian-democratic party'scandidate has already hurt a lot of feelings by claiming that "some"of those receiving compensation are "cheating the system". That mightbe true, but accusing "some" (who? me?) is obviously not the way toattract voters. This issue now has media attention and someinteresting example cases are reported.Like this one: Attorneys in Swedish district courts have beenright-sized in the past years, as part of balancing the budget. Thismeans that as soon as one gets sick, the rest get too much to do,leading to stress and burn-out, which leads to more sick leaves.Think of the court cases as HTTP requests arriving to Wikipedia.There are some processes/attorneys there to handle the cases, but forsome reason one process gets blocked and cannot work. This leavesmore work for the remaining workers, but they are probably waiting forthe first process to get finished and unlock the resources (databaserecords?) that it is using. If processes are allowed to go to sleepwaiting for each other, the work will pile up. It will never end.So, what is the solution? Throwing more attorneys at the problem?Maybe, but more likely the work processes should be redesigned andsimplified. That allows the available attorneys to finish up a caseand take on the next one. Some of their tasks are more important thanothers, but the performance or throughput of the system depends oncutting away or redesigning the most time-consuming tasks. The highdegree of sick-leave is an indicator of system design flaws (albeit anone), and thus not altogether bad.In the same way, a high "load average" (as reported by the "uptime" or"top" commands) is one indicator that the Wikipedia system is flawed.The load average in a UNIX system is the number of processes that areready to run, waiting for the CPU to become available. Unfortunately,most of them are just waiting to see if their wanted resource hasbecome available. If this is not the case (e.g. database record stilllocked), they will go back to the end of the line, waiting again. Doyou remember those bread shop waiting lines in Soviet Russia?Training new attorneys is in itself a time-consuming task, whichshould be avoided if possible. Instead of paying sick leave (for howlong?) to the already trained attorneys, a "cure" for "burn-out"should be found that can bring them back to work, thus relieving theoverload from their colleagues and saving tax payers' money at thesame time.I have no idea how a "cure" for burn-out can be found, but I think itis a necessary political trick, and thus will happen. It will nothurt voters' feelings, and it is my guess that the people who canachieve this will work for the winners of the election.This might be the weakest analogy in history, but I think we shouldtreat the Wikipedia processes with the same dignity and respect thatthe Swedish voters would expect. After all, they're supposed to workfor us. The processes feel self-fulfillment when they can finishtheir job on time, and get distressed when they get locked up. Anyuncalled for delay will only result in more work piling up. That is aflaw in the system design that has to be fixed, and we cannot goaround claiming that "some" of the workers are trying to cheat thesystem. That will only lead to us losing their confidence.-- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik Teknikringen 1e, SE-583 30 Linuxköping, Sweden tel +46-70-7891609http://aronsson.se/http://elektrosmog.nu/http://susning.nu/
4 6
0 0
links in the sidebar
by Jason Richey 04 Jun '02

04 Jun '02
I couldn't help but notice that the link in the sidebar that refers totoday's date doesn't have the spaces escaped. So, instead of clickingthrough to "http://www.wikipedia.com/wiki/May+21", you go to"http://www.wikipedia.com/wiki/May 21,2002", which puts you at"http://www.wikipedia.com/wiki/May". Perhaps this was intentional... -- "Jason C. Richey" <jasonr(a)bomis.com>
3 2
0 0
Stream-oriented diff algorithms
by Axel Boldt 30 May '02

30 May '02
The difflib code which I took from phpwiki can be used for streambased diffs (in fact, we do that: we first use a line based diff tofigure out the lines to present, and then we do a word based diff onthose lines in order to mark the changed words red.)The code works as follows (seehttp://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/wikipedia/phpwiki/fpw/diffli…):The main workhorse is the Diff class. You construct it with two arraysof strings that you want to compare. You then get a list $this->edits;this is a list of _DiffOp's which describe how to get from the firstarray of strings to the second array of strings with a minimal numberof changes. With that, you can do whatever you want. Normally, you'dpass such an Diff object to a DiffFormater (which you should extend),which then looks at the edits list and produces some output.Currently, we cut the two article version into lines, pass those twoarrays to Diff, then to TableDiffFormatter for presenting in a table.This TableDiffformatter, when printing out changed lines, calls aWordLevedDiff (which extends Diff) to color changed words in red.It might be nice to change TableDiffFormatter to produce side-by-sideoutput, similar to what the sourceforge cvs viewer does.Axel
1 0
0 0
Stream-oriented diff algorithms
by lcrocker＠nupedia.com 29 May '02

29 May '02
The next major issue to tackle with the new code is diffs. What I'd really like to find is a stream-oriented difference algorithm rather than a line-by-line one. I'm not familiar enough with the existing difflib to know if it could be used that way--perhaps its contributor could point me to some documentation on it?0
1 0
0 0
Status of new codebase
by lcrocker＠nupedia.com 28 May '02

28 May '02
The new codebase running onhttp://www.piclab.com/newwiki/wiki.phtmlis now ready for some real testing. I intend to keep this one"stable" as I add the remaining features--I have another experimentalsetup to use during development.It now has the basic features needed for view and editing articles,including redirects, article histories, recent changes, and all theuser login functions. It has the full wikipedia database from 5/20.So I'd appreciate it if you could take a minute and test the existingfeatures and tall me (1) if anything is broken, and (2) if the smallchanges I've made to aren't so small in you opinion. Don't bothertelling me what's missing--I know most of it still is.0
1 0
0 0
google and umlauts
by Kurt Jansson 28 May '02

28 May '02
Google doesn't seem to like our umlauts in page titles. Tryhttp://www.google.de/search?q=%C3%84nderungen+site%3Ade.wikipedia.com&hl=de&ie=UTF8&oe=UTF8Google says "Ungültige Seite Letzte_Änderungen. " (which means "invalidpage Recent_Changes").Is it a problem on google's or on our side? Will it be solved with thenew software?----The most important bug in the new software ontest-de.wikipedia.comseems to be that the upload feature doesn't work. All others shouldn'tbe too hard to solve, but I'm no programmer :-)http://test-de.wikipedia.com/wiki/wikipedia:Beobachtete+FehlerBye,Kurt
2 1
0 0
What happened?
by Pierre Abbat 28 May '02

28 May '02
After half an hour of trying to access Procellariiformes to see what someone else added to it, I get this:Warning: Supplied argument is not a valid MySQL result resource in /home/wiki-newest/work-http/wikiUser.php on line 24Warning: Supplied argument is not a valid MySQL result resource in /home/wiki-newest/work-http/wikiUser.php on line 32Warning: Cannot add header information - headers already sent by (output started at /home/wiki-newest/work-http/wikiUser.php:24) in /home/wiki-newest/work-http/wiki.phtml on line 48Warning: Cannot add header information - headers already sent by (output started at /home/wiki-newest/work-http/wikiUser.php:24) in /home/wiki-newest/work-http/wiki.phtml on line 50Warning: Supplied argument is not a valid MySQL result resource in /home/wiki-newest/work-http/databaseFunctions.php on line 33 What happened?phma
2 2
0 0
Database dumps
by lcrocker＠nupedia.com 26 May '02

26 May '02
I've spent more time than I care to admit loading, dumping,reloading, tranforming, testing, reloading...various wikipediadatabases before settling on what I think the new format willbe, but I made a discovery along the way that might be useful:The 05/20 database dump from wikipedia weighs in at close to600 MB. It turns out that almost 200 MB of that is cache. Inthe new system, I'll write a function specifically for doingdatabase dumps, but in the meantime I'd suggest that the nexttime you dump a tarball, clear the cache first (and don'tforget to be careful of the timestamps when you do).0
1 0
0 0
Linked list in "old" table
by Axel Boldt 24 May '02

24 May '02
>replacing the odd linked-list mechanism of>old page versions seems awkward. It could be replaced by simply>using revision numbers that would be common to both the "cur" and>"old" tables. Yes, I think the linked-list approach should go. Especially since thecurrent codebase doesn't seem to use it. We saw a while ago that the[[Pim Fortuyn]] article has a broken linked list, probably produced bysome database timeout, and yet the history page shows up fine, onlythe diffs are computed incorrectly.Every article should have a clean revision number, period. That way,there would be an unambiguous way to the n-th version of some article.The current mixture of oldid and version (look at the links on ahistory page) is a mess.Axel
2 1
0 0
php-wiki de
by Jason Richey 24 May '02

24 May '02
I have set up the test site athttp://test-de.wikipedia.com/, I had toremove an insert from the newiki.sql because it was causing an error.Of course, I lost the exact text of the error somewhere, but itbasically said that there was already an entry for Main_Page (theentry I removed looked like it may have been a discussion forum forthe actual main page). However, it was a fairly large chunk. So,take a look.A diff to my modified newiki.sql is attached.-- "Jason C. Richey" <jasonr(a)bomis.com>
7 19
0 0
Results per page:
[8]ページ先頭
©2009-2025 Movatter.jp