Serio Blog

Tuesday, 05 Aug 2008

This is a follow-up post on the subject of start-ups and acquisitions, which was posted in the middle of July.

That post looked at what happens of your support operation is suddenly faced with the appearance of a new company (which I called NewCo), and some of the questions you can ask if you are trying to make sense of how this new organisation might affect your IT support service.

This post will address the question

'how do I incorporate them [the new company] into our support system?'

...which is the question our support service gets asked most often in connection with this. So before continuing, you might want to go back to the other article and re-read the questions listed - hopefully they will add a little perspective.

1. Do you need to add the new company at all, or do anything? This might well be the conclusion you draw, so don't feel compelled to make any changes or additions to your ITSM system. It might simply be the case NewCo set's up it's own completely distinct ITSM system, and everything continues as before.

2. Add the new company NewCo as a Service Team. As mentioned in the earlier post, the issue here is one of control. Adding the IT support staff of NewCo as a Service Team (like your existing first line, network or other teams) is perfectly OK but it does imply control within the existing IT management framework, in as much as Incidents can be assigned to them, and they will be subject to the same expectations, SLAs and metrics as other Teams.

There will also be a degree of information sharing, as usually NewCo will usually be able to see shared information like your CMDB, Knowledgebase documents and ticket pools. Usually this kind of information sharing between teams is a good thing, but consider if there are any confidentiality issues to address.

3. Add the new company NewCo as a Supplier, and use Supplier Management (you'll find lots of articles about Supplier Management in this Blog).

Setting the NewCo IT support function up as a Supplier is useful if you have no control over how they operate (in other words, they are like a black box) but you still need to be able to pass Incidents, Service Requests and Changes between the two in a structured way. Setting-up NewCo as a Supplier will also allow you to collate supplier related metrics, like the number of tickets currently assigned, or assigned in the past month. Generally this kind of approach can side-step the politically sensitive issues alluded to in the earlier post. 

Friday, 01 Aug 2008

This post is for anyone who has ever been a bit confused about IP ports. Whilst most people I deal with have no problem understanding what they are and what they do, there have been a few cases recently where it would have been useful for me to refer to a post like this for background.

If you want to read about layered networks or IP in detail there are loads of resources - like here for example. I'm just going to deal with some of the practical aspects folk who work in IT support have to deal with.

TCP/IP is what is referred to as connection-orientated. That basically means it's for connecting computers together...

...except that this is not quite true. It's really for connecting applications running on computers together. And computers have more than one application running, which is one of the reasons that we have Ports.

Starting at the Beginning

You will have seen IP addresses - 123.42.1.234 for example. You also probably know that machines have names, and that names 'resolve' into IP addresses. For example 'ping zephod' might resolve into 'ping 123.42.1.234' (there is usually a server sitting somewhere on your network that takes names and returns IP addresses by doing a simple look-up from a list).

You are probably also familiar with either using an IP address to 'connect' two applications together. You might do this for example when you set-up your email client software to read your emails at home - one of the things you specify is the Name or IP address of the email server. Get that right, and the chances are you can access your email.

This IP address allows your email client software to 'talk' using TCP to your email server, allowing the two to exchange useful information because the email server is 'listening' for requests from email clients, and returning useful data back.

Ports Connect Applications

But consider this problem. That email server whose IP address you entered doesn't just run an email server. It almost certainly runs a web server as well which is also listening for requests from clients (browsers), it probably also has a database running called MySQL that also is listening for requests, and so on. In fact, if you count all of the different applications listening it may well count over 20, with the possibility of having even more.

So this begs the question: when your email client program wants to send a request to the email server application running at the IP address you specified, how does it avoid sending the data to the web server instead?

The answer is Ports

Imagine a large high-rise building with 120 flats (apartments). The address of the apartment building is 100 High St, Livingston. If you want to send a letter to a Mr Joe Bloggs who lives there, you'll need more than 'Mr Joe Bloggs, 100 High St, Livingston' because the postman won't know which of the 120 mailboxes to put the letter into. The correct address is 'Mr Joe Bloggs, Flat 25, 100 High St, Livingston'. In this example, the 100 High St part of the address is equivalent to the IP address, and 'Flat 25' is the Port number: the final piece of the jigsaw. If you don't specify a flat, or specify the wrong flat number, the chances are your letter will not be delivered.

Why don't I need to specify, or know, the Port number when I enter the IP address into my email client software?

The answer is because of convention. By convention, certain types of programs always use the same Port. So email programs typically know what Port to use. Some common standards are below:

20 FTP
22 Secure Shell
23 Telnet protocol—unencrypted text communications
25 Simple Mail Transfer Protocol (SMTP)
42 nameserver
53 Domain Name System (DNS)
57 MTP, Mail Transfer Protocol
79 Finger
80 Hypertext Transfer Protocol (HTTP)
110 POP3 (Mail)
161 SNMP
 

What about Ports used for Sending Data?

Ports are also used for sending data, but unlike applications that listen for data (like server applications of all kinds) the send Port is usually not significant. The reason is simple: when you send your request to the server application, you will send both your IP address and the Port you send the data from, meaning it's easy for the server to send a reply using this information.

(Thanks to DuncanD for his help with this)

Tuesday, 29 Jul 2008

In case you are not familiar with it, the Serio Inventory Agent (SIA) is a piece of software that runs on workstations (and sometimes servers).

It's job is twofold: provide network auditing information, and to service the Command Centre product when it wants to get information about a server machine. This post is about the first of these two, when using the Serio Inventory Agent for network auditing purposes. Specifically, what do you do if a particular machine refuses to return any information?

This post will offer a series of steps for you to follow. I'd also remind you that you don't need to use network snapshots at all to gather this data, and could instead use the file based Inventory Agent (see below).

Troubleshooting Steps

I'll call the machine you are trying to get information about (the remote computer) the Target Machine, and the machine you are using to query from your Workstation Machine.

1. Is the machine switched on? From time to time we have people call us reporting that they can't get data from a particular Target Machine, when the Target is either off or otherwise disconnected from the network. Sometimes it's because the IP address that they think the machine has is not the one it actually has, and their test Ping is actually pinging another machine. Regardless, don't overlook this simple check, and remember you can get info when a machine is switched off by using the file based Agent.

2. Is the SIA installed? Is the SIA actually installed on the Target Machine (we've had a number of cases where it isn't, but users were trying to query it anyway). Go to the machine, or use Serio Remote Desktop. Go to Add/Remove Programs (aka Programs and Features for poor souls using Vista) and see if there is an entry for Serio Inventory Agent. If not, install it.

3. Did the SIA install correctly? On the Target Machine, can you see a Service listed in Services (accessible via the Control Panel) called Serio Inventory Agent? If not, de-install and re-install. If it won't install, check the Event Log as described in Step 5.

4. Is the SIA Service running? Using the Services control panel applet, check the status of the Serio Inventory Agent Service on the Target. It should say 'Running'. Then check the Task Manager in windows to see if something called 'SerioAgent.exe' is running as a task (they are the same thing). If either of these is not shown, re-start the SIA Service, and then proceed to Step 5.

5. Is there any information in the Event Log on the Target Machine? Normal Windows applications can just display a dialog box when something goes wrong. However, for Services it's a little more tricky - Windows doesn't allow Services to do this. Instead, error messages and status information is written to the Event Log. To read these messages open the Control Panel, click on 'Administrative Tools' and then double-click the Event Viewer. You'll find messages from the SIA (Source - Serio Inventory Agent) grouped under 'Application' (Vista users: Windows Logs/Application). You should see messages like this from when the SIA was last started:

Serio Inventory Agent startup data: Port=17070, MultiHomedIPAddress=.

This message means that the SIA is listening for requests on Port 17070 (yours may say something different, like 161) and that no multi-homed IP address is set (that's usually fine, as most computers don't have multiple network cards). Make a note somewhere of the Port Number you are using.

"Serio Inventory Agent" started successfully.

This means that all startup and initialisation finished without error, and the SIA is ready. If you don't see such messages, you will probably see error messages instead. If you see one that says

The requested port is in use - is there another SNMP agent application running on this machine?

...it means that you have what is called a Port Conflict, and nothing is going to work until you resolve it. Almost all the problems we see with installs are Port Conflicts. If you see this message skip to Resolving Port Conflicts below.

6. If everything checks out OK on the Target Machine, let's run a few checks on the Workstation Machine. Run the Serio Workstation Explorer (SerioWe.exe, you'll find it in your Serio directory). From the Edit menu, select 'Edit Advanced Settings'.

Grouped under Network Options you'll see two numbers:

IP Port used for sending requests

IP Port to send requests to on remote machines

The key value is the second one: IP Port to send requests to on remote machines. This value should match the Port Number noted in step 5. If it doesn't, then change it and click OK.

Then try to connect to the Target Machine using the Workstation Explorer. If it works then the SIA is functioning just fine and answering requests, else skip to step 7.

7. If you get this far, and you've faithfully checked all the things listed, it might be time to place a support call - but be warned, this is exactly the stuff we'll check all over again.

Resolving Port Conflicts

If you get here, it's because the SIA is stuck on a Port Conflict. I won't go into the background of TCP/IP Ports, except to say this: IP Ports are like toothbrushes, and are not designed for sharing. You can do only one of two things to resolve a Port Conflict:

1. Stop the other Application.

Stop the application that is running on the Target Machine that is using the Port you noted in Step 5. If you are not using this application and it was installed by default, stopping or de-installing is usually the easiest thing to do. Finding out the identity of that application can be tricky, but if the Port in question is 161 there is a good chance it is the Microsoft SNMP Service that is using it.

2. Change the Port number that Serio Inventory Agent uses.

Tip: Only do this if you can do it for all machines - otherwise, having machines using different Ports will cause you an administrative migraine.

To change the Port used on the Target Machine, you can either de-install and re-install, or you can simply edit this registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SERIOIAGENT\

Parameters\SerioAgentStartup\SendPort

and set it to the DWORD value you require (17070 is usually a good value, make sure you set it in decimal). The value is applied when the SIA on the Target Machine is next restarted.

Having changed the Port Number, go back to the Serio Workstation Explorer and edit the 'IP Port to send requests to on remote machines' value, and then try re-querying the Target Machine.

Friday, 18 Jul 2008

I'm asked to write about how Group Companies are incorporated into overall IT Service Management. Specifically, what happens if your own company either buys another company or starts another company that will be run as a separate entity (with it's own capital, directors, and premises)?

Of course, I can't answer the question for you - but what I can do is consider some of the issues you might have to work through. Some are conceptual, some practical.

Sometimes we at Serio are asked 'how should we incorporate them into Serio?' when the real question is 'How do they fit-in in terms of IT Service Management'.

I'll refer to the newly acquired company, or the new start-up company, as 'NewCo'.

What is the 'Strategic Vision' for IT in the whole Group?

There may be a Strategic Vision that outlines the approach you take - these usually originate at group Director level. It may be that this will guide you on what you need to do (for instance, many companies centralise IT services, others have a horror of 'Head Office' services and want everything as close to the business and customers as possible). If you are lucky there will be such a guidance, but there is every possibility there won't - at least in my experience.

Will NewCo Management want control over their IT?

The directors of NewCo will be focussed on creating or developing their business, and may see IT as a critical component of their strategy. If so, any suggestion of incorporation of their IT service delivery into existing ITSM services might be unwelcome. Remember that it isn't necessary for them to be tightly integrated in terms of IT to be able to send and receive emails, and share files, with the rest of the group - you can set-up web portals specifically for this purpose.

Will NewCo have its own IT Service Management function?

First of all, is there any NewCo IT Helpdesk or Service Desk? What you are trying to assess is if there is any local function to integrate in the first place. The important thing is how such a service is perceived by the directors of NewCo.

How much interaction between your existing IT service management function and NewCo will there be?

...or phrased another way, 'how much will NewCo use group services'. Try and imagine how many support tickets might need to be passed between the two groups - but be realistic. How much will NewCo and your own ITSM group need to work together (be realistic, and resist the urge to over-estimate).

What are the politics of the situation?

This is most poisonous aspect, particularly in cases where NewCo is an acquired company rather than a start-up. As soon as anyone mentions 'incorporation' or 'merger' of IT services people see their jobs and (most importantly) their prestige to be under threat, any hope of co-operation can go out of the window. People start firing-off their CV's, attitudes become negative, it gets harder to achieve any kind of consensus. Alternatively, you might find two groups of people whose skills complement each other. 

Monday, 14 Jul 2008

As some of you are aware, we've recently (as of Friday) relocated to new offices here in Livingston. BT's technology doesn't seem capable of allowing us to keep our old telephone numbers even though we've moved no more than 2 miles to Uphall Station, so we now have new telephone numbers which are:

Tel: 01506 438855
Fax: 01506 438868

Wednesday, 09 Jul 2008

This is a follow from the post last week, continuing the topic of Team and Agent metrics.

Quality of Completion - An interesting an often overlooked figure connected with Incident Management. How many tickets are re-opened after closure? Whilst it might be that there are legitimate reasons for re-opening a closed ticket, a high number might suggest issues with quality control (closure before it's really closed).

First Time Fix Rate (FTFR) - For call-handling teams, you can look at this figure both by Team and by Agent - either with examining actual results achieved, or by looking at a 3 month trend.

Speed of Resolution - For Helpdesks and Service Desks you might have underpinning contracts and might want to examine performance from a time perspective - either resolution against target or response against target. These can often reveal more insight into why your overall service targets are hit or missed.

Backlog or Assignment Status - Something you should probably be looking at at least weekly (and ideally daily) is a status report that shows you where tickets are currently assigned - both by Team and by Agent (you'll find such a graph under performance in SerioClient). If you see relatively large numbers against a single Agent or Team, you can take action by re-distributing or allocating other staff to the Team. 

Friday, 04 Jul 2008

I'm asked to address the subject of how you measure the performance of Helpdesk/Service Desk Teams and Agents - something that can be both useful, vexatious, misleading and controversial (and possibly all of these at the same time).

If you are asked to look at this subject, you need to be clear about what aspect of performance you want to look at - saying 'I just want to look at standard reports' usually means you haven't thought clearly enough about what you wish to examine, or what problem you are trying to solve. For example, are you concerned about a Team that seems to take a long time to resolve Incidents? Or a Team that seems to re-assign a lot of tickets that are assigned to it? Do you just want to get an overview?

Let's take a look at some different aspects of Team and Agent performance. You'll see that these are mostly subsets of overall ITSM service delivery performance.

Type of Ticket - Are you looking at Incidents, Problems, Service Requests or Changes? Or all four? You might find it helpful to consider each individually.

Rate of Resolution - Now by this I don't mean speed, I simply means out of how many tickets logged or assigned to a Team did that Team actually resolve? This may not be appropriate for all ticket types and for all teams - but for some it will be. The most obvious is a first line support team, and instances where you are trying to improve the number of tickets resolved without being passed to other teams. It might be hard to determine what the 'right' or 'acceptable' number is, so try looking at the figures over something like a 3 month period.

Number Resolved - This is probably the ultimate blunt instrument, so use with caution (yet bizarrely it's often the first measure managers look at, especially for Agents). The reason it's a problematic measure is that those Teams and Agents having more complex workloads and tasks will usually have a much lower rate of resolution than those with simpler tasks. One useful thing you can deduce from this is if a particular Team or Agent is being put under increasing stress by increasing volumes. You can also investigate issues where ticket closure figures are low for one or two Team members - but remember it might simply be they are doing other things, or handling more complex cases.

Churn - Here I'm referring to an overview of incoming tickets, the number resolved, the number outstanding and the number on-passed to other teams. You'll find a report in SerioReports that shows this, and can be used to get a view of how Incidents move between teams.

I'll post the rest of this article next week. In the meantime, I've made a companion spreadsheet to our Service Desk Metrics White Paper which contains some actual examples. Search this site for downloads and you'll find it.

Friday, 06 Jun 2008

In case you have been sitting on a desert island recently (and if so, lucky you), you'll have noticed that Microsoft have announced Windows 7. The public and press reception is probably summed up by the word 'underwhelmed' as shown here and here.

What caught my eye was Steve Ballmer saying 'Vista isn't a failure'. Oh yeah? I think even having to say that tells it's own story...

As someone that works for an ISV (Independent Software Vendor) I can tell you than new Operating Systems are like Acts of God like floods, typhoons or earthquakes, but with advance warnings - there's no point in moaning about it, you just need to prepare as best you can.

It's just that, with Vista, it was really hard to see what it gave anyone that was extra. OK, it looks much nicer, but it needs a relatively big, beefy computer to run it, and in many aspects (such as networking) it doesn't interoperate well with XP or Windows 2003, and most importantly doesn't run a lot of applications (derisively called legacy applications by Microsoft) that users want to run.

It seems most of you agree. Here at Serio, only our smallest customers have taken-up Vista, and for these it's not so much a choice as it is what came pre-installed on their newest computers. For companies with more than 300 computers it seems like Vista offers nothing in return for the considerable cost and time in upgrading. What evidence we have for actual use of Vista is discussed here and it's not particularly good reading for Microsoft.

So what about Windows 7 then? Well as far as we can tell it's all about touch - rather than exclusively using a mouse to interact with a computer, you'll directly interact with the screen as you do with devices like the Apple iPhone. We've been here before though, as we blogged about over a year ago.

Personally I take touch screen with a pinch of salt. For a start, it takes a good deal of energy to hold your arms directly in front of you (go on, try it), particularly if your company's health and safety police have moved your screen to the optimum distance away from your face. I can't see users doing this for more than a few minutes before they reach for the mouse again.

Secondly, a lot of people eat at their desks. Muck that is currently sitting on keyboard and mice will end-up on screens as well where you have to look at it.

I have to say that, right now, it's hard not to feel underwhelmed by the prospect of another desktop Operating System. I see Microsoft's need to release one (investor pressure) but I can't see a corresponding user need just yet. 

Wednesday, 04 Jun 2008

This is a follow-up to this earlier post which focused on some of the downsides of running in extended hours. This post however will take a more positive stance, with some of the things that can help ameliorate some of the points made in the earlier post.

Examine your budget, because it probably isn't enough. Lots of factors will come into play such as

  • Recruitment Agency Fees (if you use them). Negotiate a realistic rate and refund period, and plan for higher than usual churn.
  • Training budgets. Inevitably higher staff turnover has a double-whammy effect: it reduces your call throughput (less experienced staff), and increases training charges.
  • Don't spend your budget exclusively on bonuses and allowances, it might not be getting you the best value.

"We under-budgeted, or rather budgeted like it was 2 x 9 to 5. The project suffered for months as a result."

Consider the practicalities. Some staff, those without cars, find travelling at unsociable hours difficult. Whilst at the start of their employment in the bright summer months standing waiting for a 10:00pm bus sounds OK, it might not be so nice in the middle of December. See if you have the funds to offer assisted transport - taxis, minibuses and the like. This can help to reduce staff fatigue and churn.

"The change having the free minibus made on the 6:00pm till 11:00pm shift was significant. It was particularly appreciated by women who no-longer had to travel on late night buses, well worth the £6k a year it cost us."

Check how you are selling the jobs. There's no point in painting an overly-rosy picture. If shift working is unavoidable don't make it seem otherwise. If you are using an agency make sure you understand that your objectives and theirs are not necessarily the same - so make sure you have seen how the positions are advertised. If you like, make a test application to see if potential recruits are being soft-soaped.

Offer a career path. This is closely linked to 'how you are selling the job' above. If you offer this as a lure to fill shift-based you need to be seen to deliver this quite quickly (but remember it's a process rather than an event).

"Setting aside some time for training, and having a structured approach, is key. Training for actual qualifications (technical qualifications like MSCE or ITIL) is useful if you can afford it. The worse the job is, the more you've got to have this. Don't forget it's useful as well because you end up with better-qualified people talking your customers."

Flexible working. It's easy to see flexible working as another complication, but it can help in service delivery as well.

"Having had constant problems with retention, we created a pool of staff. We agree to give them a minimum number of hours each month, and I can then call and ask them to come in at different times. So it might be someone's week off, having worked 12 hours the previous week, and I say ' can you work tonight for 5 hours'. If they can they say yes, if they can't they say no. Then there are others who work longer and more regular shifts. I mix the two up to get a full rota.

It's been useful for women trying to get back into support or customer-service type roles after having children. They like the extra income it gives them, and makes childcare less of an issue if they are working evenings. Some people employed this way have been with us over two years. It was a great innovation".

Thursday, 29 May 2008

If you are asked to set-up a 24-hour support operation, where do you start?

I'll leave Service Level Agreements aside for now, and assume they are in-place. What I'm going to focus on here are staffing issues and staff costs.

There are really two different cases to consider: greenfield site, and an existing operation moving from a more normal 9 to 5 or 8 to 8 to a 24-hour or near 24-hour (for want of a better expression, I'll call this a brownfield site).

Out of these two, the greenfield site seems to easier - you can recruit staff who understand what they are getting into, rather than trying to persuade existing staff to change existing shift patterns. Except, it's not as easy as that, as the comments below make clear.

As part of the preparation for this post, I called a few service delivery managers with 24-hour experience (2 running genuine 24 hour operations, 2 having once run near 24-hour operations, all running Mon-Fri only), to ask what their issues had been, and are currently. Not scientific, but useful none-the-less.

Support staff morale and 'burn' was reported, not surprisingly, as the biggest problem. Some of the comments made are as follows:

"Even when you advertise that the job is shift work on an IT Helpdesk, it doesn't seem to sink-in for a lot of people what it actually means. They just see the 'IT Helpdesk' words and see an opportunity for a career change. Some people are just fine, but others get cranky within a couple of months and start to talk the job and the company down. You're better off letting these people go in my opinion."

"Money is a rotten motivator. Extra cash for working on Wednesday to Friday only works for a couple of weeks, after that sick leave goes through the roof."

"My most important time is still 9:00 to 17:00 but moving support to later and longer hours degraded the service for the core times as well, because the most experienced people left within the first 6 months."

"My biggest mistake was the budget. I had budget to include salaries and a bonus payment based on unsociable hours, and that was it. But each time I recruit someone my agency fees are well over a thousand pounds, and their refund interval is short. My company had a corporate agreement with these guys so I was stuck with them."

"New staff were recruited by our HR department and sent to me 'to save my time'. The new starts became belligerent within the first weeks. It turned out HR was downplaying the shift work element from 'this is a shirt-work job' to 'some shift-working may be required'. This allowed them to tick their box and left me looking like a chump."

"In my case training became a difficult as the people capable of giving the training we not working shifts, but the trainees were. I could devise no solution to this problem."

"In my case we bid for the contract at too low a cost. It was like running a marathon with a broken foot from day 1. Horrible."

I'll follow this post with some of the remedies that have been successfully applied, asking if their were any positives from the experience, and expand this topic buy considering budgets.

Pages