Main Page

<grid mode=fluid>

Issues

Unanswered Questions

Goals

[edit]

80 issues.

Issue	High Level					Technology								Customers					Time		Symptom or Cause
	Product Strategy	Platform Strategy	Culture	Process	Department Alignment	APIs	SOA	MediaWiki	Caching / Storage	Performance / Scaling	Build / Dev / Deploy	Analytics / A/B testing	(Isomorphic) JS	3rd Party Installs	3rd Party Clients	Operational Burden	Maintenance Burden	Development Burden	3-12 months	2-5 years	Is Symptom	Is Cause
We need a coherent, understandable set of documented APIs for our own projects and 3rd party API consumers (an example: https://developer.github.com )						X									X						X
MobileFrontend does not faithfully display all content as the desktop experience does																					X
We need a generic storage for long term storage of content other than article renders (feed content, edits-in-process, collaborative sessions, translations, annotations, conflict resolution, media, archival renders of pages, etc)		X					X		X													X
We have duplicated functionality in core and in services (ex: title handling code, parser tests syncing, some parts of lang. variant support)							X	X								X	X				X
Configuration complexity: Installation and deployment of mediawiki + services requires a lot of different configuration files to be managed (sometimes with duplication, ex: sitematrix code in parsoid and in core)							X	X			X					X						X
Healthy extension ecosystem has the unfortunate side-effect that few third-parties are actually running "mediawiki as WMF uses it", and lots of valuable functionality is hidden in overlooked extensions		X	X					X						X			X
Wikitext parsing is tightly entangled with a lot of core code		X						X
Lack of clarity around what functionality and features are specific to the wikimedia usage and what is generic for all mediawiki users		X																				X
Code modularity (good) gets confused with "optional features" or even "unmaintained community features", since the same mechanisms are used (ie, pushback against unbundling the parser "for greater modularity" since "it's not an optional feature")		X	X														X	X			X
We need to agree on what constitutes the MediaWiki core platform and if there are additional optional bundles of important functionality		X												X								X
We need a Roadmap for the MediaWiki core platform and any important optional functionality		X		X	X			X			X			X
The third-party community needs to be involved in the development of installation/deployment strategies targeting their environments to be sure that solutions fit the problem														X
We need a better contract/set of expectations w/ third-party community, so we have a coherent lifecycle from "started by third-party" -> "adopted by WMF" -> "deprecated by WMF" -> "maintained by third-party" -> end-of-life (cf Semantic MediaWiki, PDF export, etc)						X								X	X
It should be possible to support page composition at the network edge without requiring that ordinary page design changes require source code edits across a service/language boundary						X			X				X
Third party community keeps asking for composer for extensions deployment; we should agree on a mechanism that meets their needs but does not bring composer's issues re: extension management														X
We need a mechanism for delivering assets (JS, CSS, templates) that generates the asset graph and bundles from the source code itself, rather than one defined and maintained by hand alongside it.																		X
We need a build system that supports pre-compilation of assets (JS, CSS, templates) during integration rather than immediately prior to delivery in order to support modern tooling.																		X
We need a system for defining and running A/B tests that run in isolation from those defined by other teams and that don't require source code edits to initiate.																		X
We need a system for capturing analytics events that doesn't require synchronisation with Analytics Engineering. (see also: https://wikitech.wikimedia.org/wiki/User:Ottomata/Stream_Data_Platform)
It should be possible to define and render UI components on the client and server without expertise of two distinct languages and stacks.		X														X
We need to maintain a services stack that is independent of mediawiki to deliver data that is not mediawiki related						X															X
Too many Readers technical and product initiatives (e.g., the native apps, marvin, MCS to some extent) do not have much (or any) Technology department buy-in.	X		X
In turn, there's an increasing (and unfortunate) tendency for some product teams to self-isolate.	X	X	X																		X
We don't have a cohesive engineering culture.			X		X
We don't have any consensus on whether it's valid or valuable for WMF to build user-facing products that are not MediaWiki	X	X	X		X
Certain longstanding MediaWiki technical guildelines (that it run in a shared hosting environment, that it not require JavaScript) are too conservative and are seriously impeding modernization effort		X	X					X					X	X				X
Investment in native device products has a questionable return on investment vs a single responsive, progressively enhanced, device and vendor agnostic web experience	X															X
We need consistent localization practices, well-integrated with translatewiki and our other language infrastructure
We need to erode the hard barriers between template, lua module, gadget, extension, skin, and core code to make it easier to develop and deploy code written by the community		X				X					X							X
We need to reduce our dependence on WMF-proprietary wikitext and allow non-WMF wikis to use whatever markup they prefer (this is a controversial opinion, I know)
We need to reduce our dependence on wikitext and better accomodate non-textual content								X
If we are investing time in native apps, non-WMF users should be able/encouraged to customize & use those apps as well. We need community participation in all of our code bases.	X													X
Third-parties should be able to reference and use Wikidata in the same way they can reference and use Commons															X
We need a better "innovation" process that lets teams create MVPs of disruptive products (ie, the mobile app experience, services, new skins, machine learning, etc) without creating long-lived splits and silos				X	X											X	X
We need a larger more independent community of developers (currently 78% commits over past two years from WMF/WMF-DE; compared to eg 10% employee committs for Automattic)														X
We need an asset delivery system that is easy to use for developers. Generating the asset graph from the source code would help with this, but build steps could hurt this unless care is taken to make it transparent.											X							X			X
Metrics/dashboards around project quality (instead of just quanitity)
Better collaboration with Google (especially around Google API usage)	X		X
Identify owners for critical core features and extensions that are orphans (e.g. AbuseFilter)																X
The mobile web site should be as good as or better than the app; right now the app is much better, and I use the app because the mobile web site drives me up the wall	X	X
We should make page composition faster, either at the network edge or in a Service Worker, this would make reading (logged-out and especially logged-in) much faster																					X
The multi-DC project should become a serious priority. It's basically had only 1 person work on it for the past year or two, and we're still pretty far away from being able to run active/active		X														X
by overusing services, we are shifting a pre-existing language barrier from the edge of the server stack to the middle		X					X						X
services complicate the installation process, both for developers and for third parties		X					X	X						X		X
lack of feature parity and widespread adoption in services means we are forced to maintain both the MediaWiki and the service version							X	X								X
services result in duplicated infrastructure which results in increased server spending							X									X
REST routing could be reimplemented in MediaWiki, reducing configuration complexity, network overhead and language boundaries (see line 50, many users of services do not use anything mw related) (which services would those be? The only one I can think of is pageviews. No, we have several services: pageviews, contributors, edits , unique devices: https://wikitech.wikimedia.org/wiki/Analytics/Systems/AQS#Hosted_API , also any service that would manage meta-data (ex: reading lists) would fit this category, not just analytics-focused ones)
we should investigate how to better integrate Parsoid with PHP, to reduce system boundaries +1		X					X	X
replacing Cassandra with ParserCache and moving the changeprop mechanism into MediaWiki would significantly reduce server costs and the functionality would be useful for many third-party extensions		X					X	X
resourcing of the various interfaces is not commensurate with their actual usage
Readers teams rely on various core functionality but do not take up the burden of maintaining it		X		X	X
making MediaWiki less useful for third parties cannibalizes our ability to attract volunteer developers, yet this consideration is routinely ignored
to preserve coherence as a software framework, services should extend from MediaWiki (e.g. have a REST router in core then override it with an external service where it makes sense), not live in a parallel dimension		X				X	X	X
having multiple systems with different deploy cadence (apps, MW core, services) works fine for greenfield development but increases coordination costs of maintenance											X					X
MediaWiki has a mature staging system (beta cluster, X-Wikimedia-Debug etc), the integration of other services with that system is often lacking											X
mismatch between TechCom definition of when an RfC is needed (strategic/cross-cutting/hard to undo) and how some Audiences teams interpret it (no RfC if the technical changes are not large, even if it has a serious strategic impact)				X	X																	X
lack of clarity on the status of various projects which have been tagged as RfCs but never had an official decree or discussion				X
lack of clarity around new frontend plans (e.g. how will the desktop look and who will maintain it?)	X			X	X
mismatch between how Technology and Readers understands the role of RESTBase					X																	X
developer tooling of MediaWiki is underresourced; developer tooling of most other things is pretty much nonexistent																					X
APIs are about interfaces not languages, so build a REST interface in PHP and use Cassandra if needed, but make a cache-able, powerful API that slices the data according to every use case (probably "common" rather than "every". Universailty is a long tail of things)						X
Archivability of our projects as they evolve: do we continue to rely on archive.org storing rendered pages? What's the age cutoff for maintaining an editable source representation? Will we ever solve the "rendering an old page with new templates" problem? (Another way of stating the problem: "how do you render old revisions in the face of changes to wikitext and templates--and what tooling do we need around that issue?")	X
Is "one linear chain of revisions" the history and future of the platform, or will we eventually need to add branches/fork/merge (or somethine else) to core? What about collaborative real-time edits w/ finer-grained multi-user edit histories?		X
How are media integrated in our platform: can we do something better for page layout than what we've got? Can we better support non-wikitext content? (cf https://wikimania2016.wikimedia.org/wiki/Discussions/Rethinking_the_layout_of_Wikipedia_articles https://phabricator.wikimedia.org/T90914#2918931 etc)
Can we lower the "template editing"/"module editing" wall between editors and "power users"? Are there "simple templates" we could edit/author in Visual Editor? (cf https://phabricator.wikimedia.org/T114454 for one idea)	X
General theme: enhancing commonality while improving shared functionality and enabling innovation & experiments across various device form factors (desktop / laptop / tablet / smart phone / dumb phone; online, intermittently online, offline) for both content consumption and creation
Lack of strong module boundaries in MediaWiki - anything can access anything, the skins are highly intervened with the internals making it hard to use modern frontend tech with clear division between backend/frontend code		X				X											X	X
Lack of environment to quickly prototype/test MVPs without going through the full burden of deploying an experimental feature to production
Lack of clarity of whether the product is MediWiki or wikipedia is bringing the lack of clarity in the whole architecture		X			X	X								X			X	X
Inter project event propagaton and storage updates make it difficult to efficiently display and update caches for renders that contain data from different proejcts		X
Wikidata queries (especially location based ones) take much to long to execute for end user cleints to use them		X				X
It is hard to see dependencies in phabricator and discus as different people/teams create task trees differently and there is no way to actually mark a task as blocked by another. i.e. phab is conflating substasks with dependencies
We need better dependency tracking and updating/purging artifacts when needed. Links tables and the job queue don't scale, especially not for scacading updates. For a rough idea of what that could look like, see https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/DependencyEngine		X					X	X	X	X						X	X	X				X
Using modern isomorphic JS technology (e.g. React) or similar things is hard with our stack. More and more web development happens in JS/node, so that's where all the modern tooling is (and what new hires are used to working with)		X
Our edge caches will run out of hot storage if we have an increasing set of finer grained and customized API and page responses with quick response time requirements									X
Our (MariaDB SQL and Cassandra NoSQL) databases will run out of storage or suffer performance or integrity problems if we continue adding application enhancements									X
Our event logging data stores may run out of space if event logging load increases non-linearly									X
We will run out of disk space if there's explosive growth in large files									X
Resourcing of service development is now augmented by front end engineers, which natuarlly gravitate towards JS. Moving to pure PHP is possible, but that would mean a responsibility shift back tot tech for APIs OR a change in hiring strategy for audiences		X	X	X	X	X	X	X			X							X
What responsibility do we have for the security of our platform? For protecting the privacy of our users?

[edit]

Coming Soon

[edit]

Coming Soon </grid>