Thursday, June 11, 2009

Features attract users; Documentation required to keep them

The Perl development community is, contrary to some reports, still very active and bringing out many innovative and handy modules. To keep growing, we need to get more people using Perl who are every-day developers - by which I mean, they consume CPAN modules, rather than create them.

Unfortunately, too often I am seeing users getting turned off Perl because the CPAN modules I direct them to are too hard to figure out. In turn, this puts the users off Perl. They really want to use the new modules, since they've heard good things about them, but they're not going to go so far as to join IRC and/or read the source code to figure stuff out.

The problem is that many of the modern CPAN modules don't seem very well documented.
There are two problems:
1) API is unstable / Documentation out of date.
Often tutorials were written against older versions of modules, and if users try to run them, they now receive errors or big warnings saying the feature is deprecated. This is confusing and off-putting for a new user. Sometimes even the documentation that ships with the module is out of date too!
2) Documentation is up to date, but hard to find.
One of the biggest complaints I hear about modern CPAN modules is that in order to find the docs for something, you already have to know the answer. This is due to the tendency to use OO and mix-in inheritance, and to document features in the module where the code exists. While that makes sense to a developer, a end-user will look at the other end of the inheritance tree, where they expect the feature to be exposed.

Developers don't like writing documentation, generally. But if we want Perl to increase the number of developers using it, someone will have to do it..
It's not enough to just tell users to "buy the book".. It's a pity that it takes a commercial incentive for good documentation to get written :(


  1. The standard answer will be that "be happy with the fact that there is free code". I can understand that the developers don't have the time or interest in writing documentation but this does not help in the actual problem.
    The solution could be that other people - users of the modules - help in improving the documentation. This will both share the workload and in many cases a user can write better user documentation than the original developer.

    AnnoCPAN was created to make it easy to contribute examples but one could also go to the source and send documentation patches.

    You mentioned that you pointed people at modules and they were disappointed. Would you care listing some of these modules? That would give others at least a starting point what needs to be improved.

  2. I realise that we shouldn't have any expectations, for free code.. However if Perl wants to compete with Java, Ruby, et al, then we can't have that attitude.

    I send in documentation patches when I have time, but it can be hard to know you're correct when you don't fully understand the code! I think that a developer working with a user would be the best combination, rather than a blackbox interpretation.

    Some modules that seem to come up often as unfathomable by otherwise competent coders are Catalyst and DBIx::Class -- probably because they are popular and used more -- and mostly recently, I was personally troubled by HTML::FormFu's labyrinthine docs.

  3. Part of the problem is that it's hard for module authors to write the kind of walkthrough docs you're talking about. They know the project too well, and aren't easily able to imagine what's confusing for someone coming to it fresh.

    Also, writing these sorts of docs (as opposed to API docs) is a _huge_ amount of work, much much more than simple API docs.

  4. One of the things you need to get into the habit of doing is sending users to the search.cpan dist page - for example shows -all- the DBIC core docs; a user who starts from there can generally find the right page even if they didn't know which one they wanted to start with.

    And neither the Catalyst nor DBIx::Class projects have the attitude you describe - the Catalyst tutorial has been fully updated and a DBIx::Class tutorial is in the works.

    And honestly, what commercial incentive? Nobody gets rich from writing a tech book these days ...

  5. Hi Matt,
    I agree that the Catalyst tutorial has come a long way, and is getting quite good by now. There's been good work on that. Apparently people still find specific information hard to find though, and I can understand that (looking at perldoc Catalyst::Manual right now.)

    My comments are based on feedback I get from other Perl developers - Mostly ones who aren't actively involved in any of the Perl Community. They don't join mailing lists, they don't post bugs on RT, and often they will only use the versions of Perl modules shipped with their OS.

    I think people like that make up the silent majority of Perl users, and if they can't understand the documentation and tutorials, then we *do* have a problem..

    No, I don't really know how to fix it myself, since I'm terrible as a technical writer.. but we should be aware of it.

  6. This comment has been removed by the author.

  7. Thank you for writing about this Toby. You inspired me to write a response over on my blog: