Building a natural language system for Wikipedia
The Wikimedia Foundation is the non-profit media organization behind Wikipedia. Part of Wikipedia’s push for wider use includes Abstract Wikipedia, which provides a language-agnostic version of Wikipedia articles by utilizing natural language generation to provide content in supported languages.
Overview
The Wikimedia Foundation is the non-profit media organization behind Wikipedia. Part of Wikipedia's push for wider use includes Abstract Wikipedia, which provides a language agnostic version of Wikipedia articles by utilizing natural language generation to provide content in supported languages.
Wikifunctions is being created to help facilitate process of building this multi-lingual function-as-a-service platform utlilizing open access principles found in other Wikimedia wikis.
Services Provided
This Dot Labs' engineers assisted Wikimedia by building functionality into the Wikimedia platform such as a graphical editing interface for creating, editing, and sharing functions.
Since this is a platform used by contributing site visitors, it needed to be flexible enough to handle unique types of data, and robust enough to enforce the strong type system required by Wikifunctions.
- Provided a full audit of the existing code and provided structural improvement to support scalability.
- Broke down components to support reusability and improve reliability by adding unit tests.
- Designed and developed a highly performant VueX store to effectively support the multitude of data being handled by the application.
- Enhanced the overall code structure to improve the speed of the application.
Tech-stack
Vue
VueX
Webdriver IO- Vue to provide a fast, reactive, and interactive user interface experience to users.
- Vue Test Util improved components structure and reliability by adding a complete set of unit tests.
- VueX is a state management tool used to abstract business logic from the component and handled the communication with the API layer.
- WebdriverIO created a set of E2E testing paths to speed up development by reducing risk of regression.
This Dot Labs is a development consultancy focused on providing staff augmentation, architectural guidance, and consulting to companies.
We help implement and teach modern web best practices with technologies such as React, Angular, Vue, Web Components, GraphQL, Node, and more.
Wikimedia Foundation is the non-profit organization supporting the free knowledge sharing platform, and world's most popular online encyclopedia: Wikipedia.
Properly Scoping and Sizing Tasks
The key to this project was ensuring that tasks were properly scoped and sized. While working on such a mission critical application, it was necessary to look closely at the specific tasks and requirements, understand how they fit into Wikimedia's overall goals, and then ensure that each piece of the application was developed in a way that could be measured and tested.
Strategies for Handling Custom Data Structures
The This Dot Labs' team worked with the Wikimedia team to develop a way to handle the custom data structures in a tabular format, rather than in JSON.
This allowed developers to work on individual elements of the data object piece by piece, rather than passing portions of the object around the code.
This tabular structure was placed within Vuex, a global state management library for Vue, rather than passing pieces of data through to secondary components.
Reworking Existing UI Components
With the new data structure in place, This Dot Labs' team is reworking the existing UI components (using Vue and Vuex) to utilize this new data structure.
This takes effort from the entire team to ensure that no functionality is lost. Instead, at the end, we'll have a powerful system for processing these data structures, and any changes that need to be made to them.
Covering Additional Data Types with Components
With the core in place, This Dot Labs' team was able to build out additional components for the UI to cover additional types of data, up to and including custom functions written in JavaScript, Python, and the platform-specific function composition.
These changes allow new components to be built much faster than and within months of development work, a fully working function editor was available for use.
Design
The This Dot Labs' team worked with the designers at Wikimedia to help them understand the project's complexity. They developed small sections of the designs quickly, thereby enabling the rest of the team and a small set of contributors to use the product.
The team actively worked to ensure the “complexity” of the product was actually provided by using a simple design (i.e. making something hard easy to use)! An agile and quick feedback loop was integral to the success of this partnership.
Before & After
Let's innovate together!
We're ready to be your trusted technical partners in your digital innovation journey.
Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.