Crowdsourcing Documentation at Eclipse

Crowdsourcing Documentation at Eclipse

I’ve been pensive as of late since we shipped the Eclipse Galileo release. One of the things I personally have been thinking about is how we can improve our documentation process. Currently, the Eclipse platform team has its documentation in Eclipse Help format (mostly HTML)… all this is kept in source control. In its current shape, the documentation isn’t conducive for people to contribute. There’s a lot of hoops a potential contributor needs to jump through in order to contribute documentation. Even more hoops if our documentation contributor isn’t really a developer. I also know a few committers who dread doing documentation because they have to hack away at HTML documents.

Can we do better? I think we can.

I’m interested in crowdsourcing the documentation at Eclipse.


What do I mean here? Well, I luckily have one example in the Eclipse ecosystem that is crowdsourcing its documentation already. The Mylyn project has its User Guide on the wiki.

Mylyn Wiki

The Mylyn project then uses WikiText to parse this wiki markup via a script and generate relevant Eclipse help artifacts.


In the end, there’s one source for the documentation (the wiki) where a multitude of people can easily contribute to. If we look at the Mylyn User Guide, we can see there was a decent amount of edits involving people who weren’t committers.


I’ve been experimenting with this documentation process for PDE and am pretty happy with it. There were only a couple of roadblocks I hit, but that was mostly around making it easier to use some of the WikiText ant tasks within Eclipse. I also experimented with some html2wiki transformers to help convert some of the existing PDE documentation.

One of the reasons I was inspired to look at crowdsourcing documentation for PDE was an excellent series of articles by Ekkehard Gentz on PDE’s new target platform provisioning story. I thought to myself, how could get Ekke to easily contribute that documentation (or even future docs) to PDE where it could be beneficial to everyone?

One point to bring up is that a lot of the Eclipse documentation is on the Eclipse wiki (Eclipsepedia) already. The Eclipse user community has most likely come across Eclipsepedia already. We can also assume that most people are comfortable with wiki syntax or at least the editor that comes with MediaWiki by default. If this isn’t the case, we can try to install something like the FCKEditor extension for MediaWiki to make it easier.

Another point to bring up is that having the documentation sourced from the wiki may have an interesting benefit due to early adopters. For a real world example, I’ll talk about the Equinox p2 project. There have been some complaints that the p2 documentation wasn’t robust enough for early adopters. Well, what happens if those early adopters would be willing to be part of the documentation process? As users adopt the technology, they can document their experiences along with the p2 committer team. If this was set as an expectation, I don’t think people mind sharing their experiences and being part of a team trying to build a cool technology.

What do people think? Is this good for other Eclipse projects?

Would you be enticed more to contribute to official project documentation if it was easier?

  • Posted at 10:52 pm, July 13, 2009

    I think the idea of users contributing to the Eclipse documentation is fabulous. As you mentioned, it’s not easy for the community to contribute documentation. I, like many others, have been “contributing” by blogging about things I’ve learned about Eclipse technology. It would be great if we could capture this information into a document that others could improve, and benefit from.

    I would suggest looking at Google Wave as possible technology for creating the documentation. The demo they gave of shared documents was pretty impressive. If Scott is successful at integrating Google Wave with ECF, that would pave the way to having some nice tooling for editing the docs.

  • Posted at 11:39 pm, July 13, 2009

    Well, I for my part certainly would contribute more to the Eclipse documentation if it was easier (e.g. by leveraging the Eclipse Wiki like Mylyn does). Just today, I thought that some parts of EMF could use some documentation love.

    However, I am doubtful many people would jump on the bandwagon. No matter what you do, documentation is the bane of every developer. Tools like WikiText definitely can help, but you need a constant driving force to keep people documenting stuff. Maybe we should try to introduce a docday, similar to bugday.

  • Posted at 11:59 pm, July 13, 2009

    +1 for more docs in the wiki… I’ve been beating that horse for years now, trying to get the p2 people to write more documentation during a release rather than after it so us masochists early adopters can try it out and improve the docs with examples and screenshots (and bug reports where applicable).

    Really, it’s about dialoguing with your community so they’re involved and willing to participate. Get them writing about your work, and one day they’ll be submitting patches or even taking over the project so you can move on to the next challenge. 🙂

  • Posted at 12:03 am, July 14, 2009

    The only issue for Wiki based content, is getting people to learn the wiki markup. While the barrier is lower, there are still people that will shy away from contributing to it because they don’t have a WYSIWYG type editor. What you need is a way to edit Wiki pages with a WYSIWYG editor. You almost have that with WikiText’s integration in Mylyn but not good enough.

    XML suffers from the same issue, but DocBook is widely used through out the Open Source world as a common interface. There are Free WYSIWYG editors for XML files like XMLMind (free for open source use). There is also the hosted VEX editor which gives you Visual Editing via CSS stylesheets.

    I think we are on the right track, we just need to get it to the point where users aren’t confronted with Markup and get it so they can edit the way they have become accustomed to from Word and other editors. The lower the barrier to entry the better off we will all the projects will be. Documentation for many eclipse projects is in a sorry state.

  • Posted at 12:10 am, July 14, 2009

    would be great to have an easy way to contribute documentation.
    I like the comfort of wordpress to write my blogs – its much easier then using wiki (from my POV).

    dreaming: in eclipse wiki the possibility to insert documentation from a specific blog entry – per ex. an interface to the RSS and choose an entry to import.

    perhaps – as bryan says – Google Wave could be a solution to do those things in the future


  • Posted at 1:09 am, July 14, 2009


    I think this is a fantastic idea. Anything we do to make the documentation process easier would encourage much more quality documentation. I think the internal productivity is just as important as involving others. I’m one of those who dread doing documentation; not because writing the text or taking the screenshots is difficult, but because I have to do so much html editing and site configuration. And since I don’t care for any of the html editors, I end up doing it in plain text by hand. Peter, I think you may be under-estimating the likelihood of outside involvement if barriers were removed. I think users really enjoy sharing what they know, and you can see this in the number of blog posts that are out there about various intricacies of tool use. Some of the best install guidelines for many pieces of software are put together by some frustrated user out there. 🙂

    But this begs a question that is always relevant WRT Eclipse. What about IP issues? 🙂 Do they create a barrier, and OTOH, is it always OK to include Wiki contributions with automatically generated project Help bundles? Do people just have to have a bugzilla account and related agreement in order to contribute to project Wiki, and is that enough?


  • Posted at 2:53 am, July 14, 2009

    Chris agreed. A single source that can be produced into PDF, Help, Eclipse Help, HTML, etc is a good thing. DITA has always been a bit to cumbersome for my taste, and that is what I believe the original source for the platform, as well as WTP is written in. DocBook is a good middle XML ground, but again, people need to learn tags unless using VEX or some other visual editor. Wiki is the next best choice, but still not ideal yet. having FCKEditor installed would help for the most commonly used markup.

  • Posted at 8:36 am, July 14, 2009

    FCKEditor would be a great step.

  • Posted at 1:53 pm, July 14, 2009

    Chris, Bjorn already had a vision of this. He opened a bug to start the discussion:

  • Posted at 3:41 pm, July 14, 2009

    What a great idea! I wholeheartedly agree that if people can share their knowledge and feel like their efforts will result in something good, it will happen. The low barrier to entry of the wiki makes it easy for everyone.

    As David Carver noted, wiki syntax does have some shortcomings. In particular some MediaWiki syntax makes it easy to produce malformed HTML, which web browsers handle just fine but the Eclipse help system doesn’t — it expects well-formed XHTML. Also community contributions likely won’t have planned out organizational structure. It will still take project-level resourcing to ensure high quality documentation.

    To address these issues in the Mylyn project we do these things:

    * Documentation is sourced from the wiki, but rather than have it sourced with every build automatically, it’s a manual process. This allows for a team member to update the bundled documentation while carefully reviewing changes.
    * Generated Eclipse help documentation (HTML) is kept in CVS and formatted, facilitating the review process with Eclipses’ excellent diff tooling.
    * The import-from-wiki process involves validation to ensure well-formed XML content, so that errors in the document format can be corrected at build time rather than discovered by end users.
    * Bundled documentation has a section added describing how to contribute (ie: location of the page on the wiki).

  • Lee Anne
    Posted at 2:11 pm, August 3, 2009

    +1 for a wiki-like way to allow non-committers to contribute to the Eclipse docs.

    On the Eclipse newsgroups, I see lots of questions/answers come through with example code of how to do things. And I often think “gee, that would be a good code example to have in the corresponding Eclipse help topic”. And you wouldn’t need lots of wiki WYSWYG features to pick up those things: a simple swipe-copy-paste into the corresponding help topic wiki page would be a step up from what’s possible today.