Newcomer tasks  Recommend tasks to newcomers that help them start editing
|
این صفحه، کارگروه توسعه را بر روی پروژهٔ «وظایف تازه‌وارد»، که یک پروژهٔ خاص تحت طرح بزرگتر «روز اول شخصی‌سازی‌شده» است، شرح می‌دهد. این صفحه شامل دارایی‌ها، طرح‌ها و تصمیمات اصلی است. بیشتر به‌روزرسانی‌های فزاینده در مورد پیشرفت درصفحهٔ به‌روزرسانی‌های گروه توسعه پست می‌شوند، و برخی به‌روزرسانی‌های بزرگ یا دقیق در اینجا پست می‌شوند.
با نگاه کردن به این مدل‌ها (از کلیدهای جهت‌نما برای پیمایش استفاده کنید) می‌توانید به سرعت ببینید که گروه چه ساخته است:
طراحی و برنامه ریزی برای این پروژه در تاریخ ۲۰۱۹-۰۷-۲۴ آغاز شد.نسخهٔ اول در چهار ویکی در ۲۰۱۹-۱۱-۲۰ مستقر شد.
در دسامبر ۲۰۲۰، نتایجی را منتشر کردیم که نشان دهندهٔ تأثیر مثبت وظایف تازه‌واردان بر مشارکت است.برای جزئیات به این صفحه مراجعه کنید.
ما فکر می‌کنیم که تازه‌واردان باید در اولین ورود به ویکی از هر فرصتی برای موفقیت برخوردار باشند.اما اغلب، تازه‌واردان کاری را انجام می‌دهند که برایشان خیلی سخت است، نمی‌توانند کاری را که می‌خواهند انجام دهند، یا نمی‌توانند ایده‌هایی برای چگونگی درگیر ماندن پس از اولین ویرایش خود پیدا کنند.این باعث می‌شود که بسیاری از آنها بروند و برنگردند.در گذشته تلاش‌های موفقیت‌آمیزی برای توصیه وظایف به ویرایشگران صورت گرفته است، بنابراین ما معتقدیم کهصفحهٔ اصلی تازه‌وارد مکانی بالقوه برای پیشنهاد وظایف مرتبط به تازه‌واردها است.
لازم است چند نکته را در نظر داشته باشیم:
Taking those things into account, we want to recommend tasks to newcomers that arrive at the right place and time for them, teach them skills they need to be successful, and relate to their interests.
A valuable tool we have for helping tasks be relevant to newcomers is thewelcome survey, which was originally built specifically for this purpose: personalizing the newcomer's experience. We'll plan to use the optional information newcomers give about their goals and interests to recommend appropriate tasks for them.
One of the largest challenges is going to be figuring out how to gather tasks that are appropriate for newcomers to do. There are many existing sources, such as templates that call for work on articles, recommendations in theContent Translation tool, or suggestions from tools likeCitation Hunt. The question will be which of those options help newcomers accomplish their goals.
At first, we'll focus on using thenewcomer homepage as the place to recommend tasks, but in the longer term, we can imagine building features that extend into the editing experience to recommend and help newcomers accomplish recommended tasks.
Also in the longer term, we'll be thinking about ways to tie task recommendations into other parts of the newcomer experience, such as theimpact module on the homepage, or into thehelp panel.
We know from research and experience that many newcomers fail early in their editing journey for one of these reasons:
Those tasks are difficult enough that they likely fail and don't return.
We also know that on thenewcomer homepage, the most frequently clicked-on module is the "user page" module -- the only thing on the page that encourages users to start editing. This makes us think that many users are looking for a clear way to get started with editing.
And from past Wikimedia endeavors, we've seen that task recommendations can be valuable.SuggestBot is a project that sends personalized recommendations to experienced users, and is a well-received service.TheContent Translation tool also serves personalized recommendations based on past translations, and has been shown to increase the volume of editing.
For all these reasons, we think that recommending specific editing tasks for newcomers will give them a clear way to get started.For those newcomers that have an edit in mind that we want to do, we'll encourage them to try some easy edits first to build up their skills.For those newcomers who do not have a specific preference on what to edit, they'll hopefully find some good edits from this feature.
There are many terms that sound similar and can be confusing. This section defines each of them.
The core challenge to this project is:Where will the tasks come from and how will we give the right ones to the right newcomers?
The graphic below shows our priorities when recommending tasks to newcomers.
As shown in the graphic above, we would give newcomers tasks that...
For instance, we do not want to give newcomers tasks that are irrelevant to what they hope to accomplish.If a newcomer wants to write a new article, then asking them to add a title description will not teach them skills they need to be successful.
We're splitting this challenge into two parts: thesourcing the tasks andtopic matching.
There are many different places we could find tasks for newcomers to do.Our team listed as many as we could think of and evaluated them for whether they seem to be achievable for the first version of the feature.Below is a table showing the many sources of tasks that we evaluated in coming to the decision to start by using maintenance templates.
| Source of task | Explanation | Evaluation |
|---|---|---|
| Maintenance templates | Most wikis use templates or categories to indicate articles that need copyediting, references, or other modifications. These are placed manually by experienced users. | Easily accessible. Already used inSuggestBot andGettingStarted. |
| Work on newest articles | New articles may be good candidates for work because they likely could be improved or expanded. They are also more likely to be about current topics. | Easily accessible, but most new articles are created by experienced users, and may not need help from newcomers. |
| Add images from Commons | There are articles that have images in some language Wikipedias but not in others. This could be a good task for a newcomer who created their account in order to add an image of their own. | An idea with high potential, but would require a lot of work to build interfaces. There are also questions about how to identify whether an article needs an image, and which one to recommend. |
| Expand short articles | Many articles are stubs that could be expanded. | This task is probably too open-ended and difficult for a newcomer. |
| Link to orphan articles | Many articles have no incoming links from any other articles. Users could find articles to link to the orphan articles. | Easy to identify orphans, but may be confusing for a newcomer to have to go findother articles in order to do the task. |
| Add references | Many articles are in need of additional references or citations. | Probably a challenging task for a newcomer. Frequently covered by maintenance templates. |
| Add categories | Categories are used for many purposes on the wikis, and adding them to articles that don't have them could be a low-pressure way to contribute. | Newcomers may not have good judgment when it comes to adding categories. This also does not teach editing skills that they need for other tasks. |
| Content translation | TheContent Translation tool could be a good way to structure the editing experience and help newcomers write new articles without having to generate all the content on their own. | An integration here could be great -- we may want to use the welcome survey to distinguish which newcomers are multilingual. |
| Add sections | There arealgorithms in development that can recommend additional section headers based on similar articles. | Writing a new section from scratch may be too challenging a task for a newcomer. |
| Specific link recommendation | Adding wikilinks is one of the best tasks for newcomers. It would be powerful if we could not only tell a newcomer that an article needs more links, but indicate which specific words or phrases should become an link (internal and/or external, depending on local policies). | Some research has been done on this idea that the team will be looking into, as this idea could be a perfect first edit for a newcomer. |
| Copy editing | Many articles need copyediting, but it would be a better experience for newcomers if we could suggest specific changes to make in article, such as words that are likely misspelled or sentences that likely need to be rephrased. | While this would be an excellent experience for the newcomer, we don't have a way to approach this. Perhaps experienced could flag specific copy edit changes instead of fixing them. |
| External link cleanup | Help ensure articles follow external link policies. | Could be populated by theexternal links cleanup maintenance category. |
| Neutral point of view | Offer people suggestions for how they can "neutralize" subjective text (T376213) | Previous research indicates that algorithms could be used to recommend edits that enhance the neutrality of articles. |
In version 1.0, we will deploy the basic parts of the newcomer tasks workflow.It will recommend articles to newcomers that require different types of edits, but it will not match the articles to the newcomers' topics of interest (version 1.1), and it will also not guide the newcomers in completing the task (version 1.2).
We're going to be starting by usingmaintenance templates and categories to identify articles that need work.All of our target wikis use some set of maintenance templates or categories on thousands of articles, tagging them as needing copyediting, references, images, links, or expanded sections.And previous task recommendations software, such asSuggestBot, have used them successfully.These are some examples of maintenance categories:
Inthis Phabricator task, we investigated exactly which templates are present and in what quantities, to get a sense of whether there will be enough tasks for newcomers.There seem to be sufficient numbers for the initial version of this project.We are likely to incorporate other task sources from the table below in future versions.
It's also worth noting that it could be possible to supplement many of these maintenance templates with automation.For instance, it is possible to automatically identify articles that have no internal links, or articles that have no references.This is an area for future exploration.
During the week of October 21, 2019, the members of the Growth team did a hands-on exercise in which we attempted to edit articles with maintenance templates.This helped us understand what challenges we can expect newcomers to face, and gave us ideas for addressing them.Our notes and ideasare published here.
Our team's designer reviewed the way that other platforms (e.g. TripAdvisor, Foursquare, Amazon Mechanical Turk, Google Crowdsource, Reddit) offer task recommendations to newcomers.We also reviewed Wikimedia projects that incorporate task recommendations, such as the Wikipedia Android app andSuggestBot.We think there are best practices we can learn from other software, especially when we see the same patterns across many different types of software.Even as we incorporate ideas from other software, we will still make sure to preserve Wikipedia's unique values of openness, clarity, and transparency.The main takeaways are below, and the full set of takeaways ison this page:
Our evolving designs can always be found in two sets of interactive mockups (use arrow keys to navigate):
Those mockups contain explorations of all the difference parts of the user journey, which we have broken down into several parts:
Below are some of the original draft design concepts as the team continues to refine our approach.
During the week of September 16, 2019, we used usertesting.com to conduct six tests of the desktop newcomer tasks prototype with internet users unaffiliated with the Wikimedia movement. In these tests, respondents are compensated for trying out the mockups, speaking aloud on what they observe, and answering questions about the experience. The full results can be found inthis Phabricator task. The goals of this testing were:
During the week of September 30, 2019, we used usertesting.com to conduct six tests of the mobile newcomer tasks prototype.The full results can be found inthis Phabricator task.The goals of this testing were the same as with desktop, but with the added goal of understanding how the mobile experience should differ from the desktop experience.Mobile user testers were prompted with the scenario of intending to add an image to Wikipedia (whereas desktop respondents were prompted with the scenario of intending to create a new article).
Summary of findings
Recommendations
Past research and development shows that users are more likely to do recommended tasks if the tasks match their topical interests.SuggestBot uses an editor's past editing history to find similar articles, and those intelligent results are shownin this paper to be executed on more often than random results. TheContent Translation tool also recommends articles based on a user's previous translation history, and those recommendations have increased the translation volume.
In looking at the usage of V1.0 of newcomer tasks, which does not contain topic matching, we see that there are users who navigate through many suggested articles, and end up clicking on none. There are also users who navigate through many, and end up editing only the ones they happen to find that belong to a certain topic, such as medicine. These are also good indicators that topics can be valuable to help newcomers find articles they want to edit.
Our challenge with newcomers is a "cold start problem", in that newcomers do not have any edit history to use when trying to find relevant articles for them to edit. We want to have an algorithm that says what the topic is of each article, and use that to filter the articles that have maintenance templates.
There are multiple approaches with which we might find articles that match a user's stated topic of interest. While our team identified many, we built prototypes for three methods and tested them:
Inthis Phabricator task, we evaluated the three methods, and decided to proceed with the ORES model.The Growth team worked with theScoring team to strengthen the model, and with theSearch team to make the model predictions available to the newcomer tasks workflow.During the time that this work was happening, we deployed the somewhat worse-performing morelike algorithm, and switched to the ORES model about a month later.
The ORES model we use now offers 64 topics, and we chose to expose 39 of them to newcomers. The evaluation in four different languages showed that on average, 8.5 out of 10 suggestions for a given topic seem like good matches for that topic.
In designing interfaces that allow newcomers to choose topics of interest, these are some of the considerations:
These mockups contain our current designs for this interface. You can navigate with your keyboard's arrow keys. Below are some images of the mockups:
Guidance was deployed on 2020-06-15.For a guide to translating the messages in this feature, seethis page.
After newcomers have selected an article from the suggested edits module, they should receive guidance about how to click edit and complete the edit successfully.While it is exciting that some portion of newcomers are completing suggested editswithout guidance, we're confident that by adding guidance, we will substantially increase how many newcomers edit.
We decided to repurpose thehelp panel as the place to deliver this guidance.Reusing the help panel will allow us to build quickly.The guidance contains three phases:
Some of the ideas we considered implementing included:
During the last week of December 2019, we user tested desktop and mobile prototypes, which can be found below.We will post the user test results after assembling them.
Below are some images of the prototype:
After deploying the first version of newcomer tasks, we want to start testing different variants of the feature, so that we can improve it iteratively.Rather than just having one design of newcomer tasks, and seeing if newcomers are more productive with it than without it, we plan to test more than variant of newcomer tasks at a time, and compare them.We have compiled an exhaustive list of all the ideas of variants to test -- but we will only end up testing perhaps 10 per year, because of the effort and time it takes to build, test, and analyze.
In March, April, and May 2020, we'll be testing variants that aim to get more users into the newcomer tasks flow.
See this page for the list of variant tests and their results.
In December 2020, we published the results of a controlled experiment showing that newcomer tasks have a positive impact on engagement. These are our most important results, and give us confidence that these features should expand to more wikis.See this page for the details.
Starting in December 2019, we have been tracking several key metrics from newcomers tasks.The graphs shown in this section are our main charts of those metrics as of 2020-08-17.
Since deploying newcomer tasks in November 2019, we have seen steady increases in both the number of edits from the feature and the number of editors using the feature.These increases are due to two elements: (a) improvements to the feature, and (b) expanding the feature to more wikis.
Conversion funnel: the first graph is the most important to our team.Each line shows how many newcomers arrive at each stage of our "conversion funnel", meaning how far they progress into the newcomer tasks workflow, as a percentage of newcomers who visit their homepage.We want the users to move through the stages of:
In general, we want to see all the lines go up.
Edits: the second graph shows the number of newcomer task edits completed each week, with a separate line for each wiki and a "total" line in black.From December to August 17, there have been 15,126 edits completed through newcomer tasks.It is clear that this has grown over time, which is certainly to be expected because we have gone from 4 wikis to 12 between January and August.
But looking at the individual wikis' lines, it is possible to see growth over time.
Editors: in addition to tracking the number of edits, we also want to make sure that increasing numbers of newcomers are participating.The third graph shows the number of users completing newcomer tasks each week, broken out by wiki.
The Growth team's ambassadors have gone through over 300 edits saved by newcomers and marked whether or not each edit was productive (meaning that it improved the article).We are happy to see that about 75% of the editsare productive.This is similar to the baseline rate for newcomer edits, and we're glad that this feature has not encouraged vandalism.Most of the edits are copyedits, with many also adding links, and some even adding content and references.About a third of users who make one suggested edit go on to make additional suggested edits. Many also go on to make edits that arenot suggested by the feature, which is behavior we are happy to see.
The high-quality edits we're seeing encourage us to improve the feature so that more newcomers begin and complete its workflow.