More on Knowledge Base QA Systems

This is a follow-up to my last post Designing a KB Quality System. In this post, I’m going to give a more detailed description of what is included.

Recall in the last post I said we wanted

  • Accuracy
  • Relevance (conformance with the Editor’s Brief)
  • Non duplication of content
  • Feedback mechanisms
  • Periodic review
  • A way for new documents to be suggested

The first 3 of those mean that it’s unlikely you’ll have a system where anyone can create a document and just add it (what I call a dumpster Knowledge Base). Of course, you could say that contributors have to check these things before adding, but this is likely to be done with just varying degrees of success – and the bigger the team the more problematic it will usually become. (As an aside, I’m not a big fan generally of anything being owned by ‘the team’).

Instead, you will have a small number (ideally 1) of Editors who will check accuracy, relevance and uniqueness before adding. There are two ways you can use Editors:

  • The Editor writes all documents based on suggestions from colleagues or content consumers
  • Or, The Editor checks documents written by others before inclusion.

My experience is that although the second option offers the prospect of more and better content, in practice you’ll need someone to lead the process and will probably end up doing the first.

You need some form of feedback mechanism. For Serio users, this will usually mean you enable the ‘rate this document’ functionality, which allows SerioWeb users to comment on documents for you, or you simply have a special ‘feedback’ email address that goes direct to the Editor. The more technical your audience, the more likely they are to report technical errors in my experience.

Periodic review does just what it says. You need to review documents every once in a while because technology changes. For example, an incompatibility between two products might be resolved, prompting either removal or updating of the document.

Finally, you need a way for content users to suggest new documents. Now, there are two ways to do this.

  • Look at what consumers are searching for.
  • Or, Ask for suggestions directly.

Looking at what search terms are being submitted, and what results are returned, is an essential activity. If you are a Serio user, please note that logging of this data is OFF by default – switch it on and you’ll be able to see all the search terms being targeted by your consumers (see ‘Monitoring search terms used’ in the HowTo guide). This will help you identify weak areas and suggest areas for improvement.

If, as an editor, you just say ‘suggest some articles!’ you probably won’t get much of a response. Instead create a simple & structured environment. For Helpdesks and Service Desks, this usually means a way to ‘flag’ Incidents at the point of closure so as to say ‘a knowledge base article for this is required’.

For Serio users, this usually means using Agent Status B (set to something like ‘KB Suggestion’) and a question like ‘Should this Incident be suggested for Knowledge Base inclusion Yes/No?’ as part of the resolution Action. All the Editor need to do is scoop these up once in a while, and decide based on the Editor’s Brief if it should be included or not.

I’ll take all this in my next post and construct a worked example.