When the computers go down...

Re: When the computers go down...

CatherineThomas — Sat, 25 Aug 2012 09:56:14 GMT

Recently we seem to have had quite a few problems with the server going down, problems with phone lines or power cuts. Being a corporate practice we have someone at head office to call for IT support and they will usually sort things out as quickly as possible. But in the mean time we have to cope. We back up the system every night but if the server is down we can't even turn the computers on. We have blank labels with practice details etc for medications and a list of prices that we can refer to so that we can still take payments. Although sometimes they work out slightly differently when we put it on the computer eventually, most people are happy with this. The worst problem is appointments, as we don't know what appointments we have booked, particularly when the computers go down first thing in the morning, and clients are calling to book appointments. So we end up with people turning up at all times with many double booked appointments. Most clients tend to be quite understanding. Although I did have one client who was not happy when she brought her cat in for vaccination without a vaccination record. I had no idea if the cat had FeLV vaccinations or not, neither did the owner. She was not happy that I couldn't tell her with certainty which vaccinations the cat would need and therefore would have to come back.

The worst situation was when we had a power cut recently. One of the other vets had just started a cryptorchid castration in theatre. Theatre has no windows and we don't have any back-up power. So she was searching for abdominal testicles by torchlight. Not a fun experience. It was also pretty difficult trying to examine animals in consult with limited light and coping without computers too. Fortunately we got an electrician down to sort out the problem reasonably quickly. (The lights came back on just as the dog was being woken up from surgery).

I guess this shows that we really should have better back-up facilities for when we do have these problems.

Re: When the computers go down...

Ian Mostyn — Sun, 19 Aug 2012 23:24:34 GMT

[quote user="Alet Engelbrecht"] The most frustrating part is that the wifi signal is variable in one of the consulting rooms, but a minor annoyance.

[/quote]

Wifi? No, wired connection every time gives much more reliability, especially if you can double up on wires when installing them. I use wifi at home but would never use it at work.

Ian

Re: When the computers go down...

Alet Engelbrecht — Sun, 19 Aug 2012 12:10:33 GMT

I am currently working in a practice that uses a web-browser/cloud based system (a first for me). This is an old system that was designed by the owner's son, so it has limitations, but I really like the flexibility it gives. The most frustrating part is that the wifi signal is variable in one of the consulting rooms, but a minor annoyance. If the router dies, smartphones and 3G broadband connections can still provide options. Add in laptops and you can even overcome power failures!

Re: When the computers go down...

Tim Cheyne — Thu, 16 Aug 2012 15:31:30 GMT

[quote user="Sarah Keir"]

How many practices clean/vacuum the server? Ours is in a back room but I always get a shiver down my back when I see how much fluff has collected there!

See

http://www.theregister.co.uk/2010/11/26/ventblockers_2/

[/quote]

One of my lads frequently insists on blowing the fine sand out of my PC and clearing the fans. Having seen this article I am glad that he does.

Tim.

Re: When the computers go down...

Sarah Keir — Thu, 16 Aug 2012 13:28:48 GMT

How many practices clean/vacuum the server? Ours is in a back room but I always get a shiver down my back when I see how much fluff has collected there!

See

http://www.theregister.co.uk/2010/11/26/ventblockers_2/

Re: When the computers go down...

Anthony Todd — Sun, 12 Aug 2012 10:14:14 GMT

[quote user="Ian Mostyn"]

I know this is a bit late possibly but here is my computer system plan. I can only imagine how much stress you have had recently and this is my effort and preventing that stress

Redundancy, back up & testing.

Computers do tend to have a habit of breaking at the worst moments but a proper contingency plan needs to be in place. Your database and computer system is possibly the most valuable commodity in the practice and should be protected. We are by no means perfect and I am always trying to improve the system security but here is a run down of our system.

Back up - We do 2 back ups every hour to both a different disc but also a different physical machine. As soon as we get fibre optic broadband I will be sending a copy of this back up to both my home server and a back up server at the other branch. By seperating the 3 backups to three different physical sites we prevent problems if we had a fire, riot etc. If your data doesn't exist in at least 3 places it doesn't exist at all.

Secondary Server - If our prinary server dies, we can use one of the backups that we have made as the database and keep working on a secondary server. At most we can lose 30 minutes of data but we can keep working within minutes of a problem developing.

Redundancy - I am possibly a bit paranoid but we have two broadband lines with different companies and I am pondering a 4G connection as a third reserve. I have all my servers, telephones and modems etc on 2 battery backups (UPS) as well as one reception computer and an office PC. Again I could make sure all PC's are on a UPS and I will do that in the future. I have a spare network switch ready to go, just swap cables and I have at least one spare modem for the broadband. I don't have a spare router which does bother me but I have two at home that could be brought into use within 30 minutes. I have two ways of accessing the telephone lines and three separate ways of making out-going calls. My credit card machine has a manual backup as well. We have a printed price list and some blank labels ready to go. I have spare keyboards, mice, monitor and a PC that can be moved anywhere in the practice if a workstation fails.

Disaster Plan - we have a laminated disaster plan at both sites with what to do in case of computer failure, telecoms failure, loss of water, electricity, gas etc but also outbreaks of infectious disease - like swine flu. All important phone numbers are on the list and I make sure that senior staff know what to do when I am away.

Testing - Whatever plan you have you need to test it to make sure it works. We have had to use a lot of these backups and redundancies over the years and generally they have worked well but I always try to improve it. A day without the system is going to costs hundreds if not a couple of thousand. A total failure and loss of the database could cost tens of thousands.

I am glad everything is getting back on track for you and fingers crossed you are back to normal on Monday.

Ian

[/quote]

Perfect and total advice!

Buy your own spares of everything connected by plugs, a spare whatever is far cheaper than your "support" ;it's a ten minute job to change a box.

Change the HDDs, or get the server repaired later! [3 HDD drives failing simultaneously must be the rarest computer event ever???]

Restore your backup on another machine always and every day.

So called support usually sucks, and if it doesn't then it's too late and too expensive.

I'd welcome your disaster plan please, please, Ian. It may just avoid the rare, but not for us, "sh1t, I never thought of that" moment on a Saturday afternoon with a full waiting room....

There was a statistic some time ago that stated [figures from memory] "30% of firms which suffer a major data loss go broke".....

Re: When the computers go down...

Ian Mostyn — Sat, 11 Aug 2012 23:03:49 GMT

I know this is a bit late possibly but here is my computer system plan. I can only imagine how much stress you have had recently and this is my effort and preventing that stress

Redundancy, back up & testing.

I am glad everything is getting back on track for you and fingers crossed you are back to normal on Monday.

Ian

Re: When the computers go down...

Julie Innes — Sat, 11 Aug 2012 21:32:09 GMT

Ok, I'm feeling a bit better about it all now. Yes, staying at the surgery all day until 6pm (sob! And it was one of the only sunny days we've had recently!) was a pain, BUT an engineer did come out, replaced all 3 disks in the server that had gone belly-up (apparently you'd need to be "extremely unlucky" for this to happen!) AND my IT person is coming up tomorrow (Sunday) to re-install the operating system and back-up, which does mollify me somewhat (and the last few glasses of wine while watching Mo Farah's success in the 5000 have helped

So at least we should be back up and running for Monday, thank God! And hopefully our 2nd line will be fixed then too.

Thank God for wine!

Re: When the computers go down...

Nicola Lawlor — Sat, 11 Aug 2012 20:00:12 GMT

We had similar last week when our broadband router died. Ours is a cloud based system so we couldn't access any notes or prices at our branch via the usual route. Called the other branch and they printed out and faxed over our appointment list and we just wrote any extras down so we knew who was coming in. We could access history via smartphones so used this for at least double checking meds etc where necessary. We had "blank" preprinted medication labels and just wrote out the individual extra information on these, likewise had blank consent forms for surgery/euthanasias available if needed. Clinical notes were written onto paper with lists of what needed to be priced and once the system was eventually up and running again it was transferred to the notes and price up, invoices posted out.

With regard to getting it fixed - called the internet provider for info and they guided us by phone as to tracing the fault and then which piece of hardware to replace which we did. It probably helped that is was a Monday morning in getting hold of them I guess?

Re: When the computers go down...

Evelyn Barbour-Hill — Sat, 11 Aug 2012 19:01:00 GMT

[quote user="Julie Innes"]can anyone remind me again why oh why oh why I ever thought setting up my own practice would be a good thing??[/quote]

Yes, but surely you know really! Glorious freedom.

Re: When the computers go down...

Evelyn Barbour-Hill — Sat, 11 Aug 2012 18:57:33 GMT

[quote user="Stephen Courtney"]- we cannot even take money without a computer[/quote]

Wow. Sounds like a serious fault in the system. Why not? Can't you just hold your hand out and say "That'll be thirty-six pounds please, Mrs. Jones", and when she pays you...... well, maybe you don't have a cashbox, but everybody has pockets or a teapot or something.

[quote user="Stephen Courtney"] there were a fair few chancers who disappeared off the horizon and never paid their accounts.[/quote]

Now there's a surprise.

Re: When the computers go down...

Stephen Courtney — Sat, 11 Aug 2012 18:44:46 GMT

The quality of your IT support, especially at weekends and at night, is a direct function of the value of the maintenance and support contract you hold with your IT supplier.

If you have the cheapest option, you will get lousy support. If you have a good support contract that you are paying good money for, nail them - legal action can focus the mind fairly sharply. if they have let you down make them pay for the privilege, because there is only one certainty - it WILL happen again.

My branch had a similar episode a couple of years ago, when it happens we consult on paper, and put charges on later - we cannot even take money without a computer - and while eventually we got it all back on to the computerised records there were a fair few chancers who disappeared off the horizon and never paid their accounts.

Re: When the computers go down...

Julie Innes — Sat, 11 Aug 2012 15:24:12 GMT

I would need to dig out and look at the contract, but I was led to believe that there was always someone I could reach (there was, she just didn't know about my sysytem, or even who I was, so had to try and get hold of someone else.)

they told me they could have a new server with me in 4 hours, in cases like this. I wasn't made aware that WE would have to call the server provider, be put on hold for over 20minutes at a time, then have to wait around for 4 hours (I did ask if they could contact me when they were almost there but no.

I do back up every night, so no records are lost. It's just that I can't access any of them!

My dad did offer to do all my software etc for me, but it seemed like a lot to ask of him and I wanted back up in case the system went down and he wasn't around (ironic, huh?!)

I did try to charge for a few things I knew the price of, but people only seemed to have plastic, and there isn't a bank very near. seemed a bit off to insist they go and get cash considering it was our cock-up in the first place (well, not mine, but it reflects on me, think that is what I am most annoyed about- made us look hugely unprofessional this morning, and I hate that!)

Seems like the phone line problem is a different one!! My 2nd phone line has gone to the wall, and there is some fault that needs an engineer to come and fix- at 8am on Monday. Fantastic.

can anyone remind me again why oh why oh why I ever thought setting up my own practice would be a good thing??

Re: When the computers go down...

Evelyn Barbour-Hill — Sat, 11 Aug 2012 14:50:21 GMT

[quote user="Julie Innes"]1. Am i crazy to think that they should come and fix the problem directly without having to take a member of (saturday) staff out of comission, or having to get my poor dad down here?[/quote]

No. (But it depends what the contract says.)

[quote user="Julie Innes"]2. Do you all have something in place for if/ when this happens, eg written files? Seems a bit onerous to have both, and defies the object of "paperless"[/quote]

The smug answer would be "don't be paperless!" Apart from that: back up, back up, back up.

[quote user="Julie Innes"]3. Are your computer service providers better! And if so, who are they?![/quote]

Yes. Me. Smug

[quote user="Julie Innes"] couldn't charge anything or take money (credit card line died too, just to complicate matters!)[/quote]

The credit card line dying was certainly a disaster....... just coincidence, or was there a connection? (Or possibly a conspiracy by giant green squirrels from Alpha Centauri?) But couldn't charge anything? Make a guess at the right amount, you won't often be far out. Many people can fork out cash or give you a cheque if the card doesn't work.

Sorry, I do sympathise really.

Re: When the computers go down...

Joyce Whitehead — Sat, 11 Aug 2012 14:29:33 GMT

Sympathies - what a nightmare. We don't have full back up in paper, but we have an emergency "Power off/computers off" file - we are very rural and get more than our fair share of power cuts. This file has a full client list with phone numbers, some pre printed blank dispensing labels (ie just practice address and usual warnings on them), price lists, and sheets to write up cases so that they can be entered on the system once we are up and running again. Inevitably it is much harder to price things than with the computer on, but at least we can take a reasonable stab at the amount of money owed.

Although we use a computer appt system we currently also use an appointment book. We are thinking of ditching this, but maybe will keep it for now! I think you certainly don't seem to have had the best service so far...

Re: When the computers go down...

Alan Tevendale — Sat, 11 Aug 2012 14:20:54 GMT

The joys of computers. We are lucky in many respects that we have two sites and as such if things get really bad I could always draft in the other server from the branch sugery. I don't mind IT too much myself so doing that isn't going to be too difficult. Another 'backup' that I do is to use remote access links to both our servers and as such I can operate both servers from the main surgery or home (not always a good idea if you want a day off). By doing this it would take both servers going down at the same time to take us completely offline. Or one server and the broadband link.

In answer to your questions

1. Have you any prearrangement with your IT support as to what happens out of hours (their out of hours not yours)? If you don't actually have any weekend cover from them how can you expect them to help? - they might not have any staff working at the weekend.

2. We certainly have nothing else in place other than computers. I'd certainly hate to have to write up a case twice just in case the computers went bad.

3. We've used Verifac for years and can usually manage to get hold of them when we need them.

It really does sound like you've been having one of those mornings. Another thing that I find helps is that anytime we make any major price updates I print off an up to date price list. If we do have computer problems then things can always be priced up manually and the computers brought back up to date when they are working again. Just take paper notes in the meantime.

Out of interest, which computer service provider are you using?

I really hope things are getting sorted.

Alan