Friday 6 July 2018

How do Microsoft make some great programs and some terrible ones? OneDrive sucks!

So this is the first time I have ever used OneDrive, having previously used DropBox, Box and Git and the syncing is rubbish because of the way exclusion works. Anyone who has written a sync tool knows that you do not want to synchronize everything but when you first enable OneDrive, it has created a Documents folder, which has been pinned to your Quick Launch. From now on, every time you think you are saving to your documents, you are actually saving to OneDrive and NOT your documents.

Microsoft, seriously? For those of us who have used Windows for years, this is deception and just causes confusion. Why would you call the folder the same name, give it the same icon and put it where my normal documents folder is so that if I were to accidentally delete some OneDrive documents from another machine, I would accidentally come home to find I don't have any documents any more!

So eventually, I realise this after managing to delete a vendor folder that had taken the previous 10 minutes to download! So I moved my stuff back into "My Documents" and deleted OneDrive's "Documents". Now I am wondering how to sync some folders.

The instructions online are out of date, there is no option to "Choose OneDrive folders to sync" context menu item. Secondly, trying to right-click the folder in OneDrive brings up the context menu for about a second after which it disappears. I finally found out how to access the options via the system tray icon.

Vendor, node_modules and others are basically cached packages that take up way too much space and change too frequently to need syncing, they can easily be re-downloaded if needed and to avoid eating up bandwidth, I wanted to exclude a folder in my web project but otherwise sync (i.e. backup) the code part of the project.

Due to reasons that are hard to understand, however, the OneDrive documents is some kind of symlink to the real documents so if I include a folder and then exclude a sub folder from OneDrive, it deletes it from my local computer even though it was created locally in the first place! Basically, this is trash and makes OneDrive completely unusable for me.

I don't want to be a moaner but honestly, there is no way that anyone serious could have tested and accepted this product before it was released because all other products I have used have already worked this out. Ignore means ignore, not delete!

It is also super slow. Bye bye OneDrive, I will continue to use DropBox, which is to be honest, pretty poor in some ways but is still yards better than OneDrive.

Thursday 5 July 2018

The NHS at 70

I thought I would write some armchair opinion about the NHS, a much beloved and seemingly struggling Institution that turns 70 today and is under more pressure than ever. I don't have much first-hand experience, although I have worked in a few temporary admin roles in a hospital, have a mum who worked in the NHS for the last 25 years as a Ward Manager and have a number of close GP friends, so my opinions are not expert but they are not "tabloid" either!

We all love it

I think some people need to understand that everyone loves the concept of the NHS. I love the fact that with generally minimum fuss, I can get a range of benefits like free ambulance transit in emergency with paramedics, free acute care in Accident and Emergency (or whatever they might call it now) as well as a range of in and out-patient facilities, which are free at point-of-use. In reality, we get thousands of pounds worth of treatment, which for many people is way more than they have paid indirectly in taxes.

Statements like "the Tories hate it" or "the government want to kill it off" are disingenuous and miss the real problems that are occuring and most importantly, why. It is ultimately a question of money and efficiency - i.e. value for money but this too is too simple an approach. To measure the problem purely in terms of money assumes that firstly the NHS is fit for our modern world and secondly that it is largely a case of efficiency combined with extra government contribution that will fix it.

Hopefully most people will understand that it is not as simple as that!

Things have changed

Firstly, the NHS itself has changed massively in the past 70 years, partly because the world has changed and partly because the remit of hospitals 70 years ago was much smaller than it is now. 70 years ago, we didn't have MRIs and Cat Scans, we had more rudimentary X-Ray machines. Ambulances were basically vans with a gurney and a driver and for the most part, there would have been little need to visit A&E for small issues like cuts since their treatments would have been fairly close to what you would do at home: wash it and cover it.

An ambulance in 2018 might cost £250,000 or more each! That is a lot of money, especially since most of the people using it will not need most of the kit, but for the odd few people who do, it is necessary to have it on most or all emergency ambulances. Compare that to 1948 when the price for a basic van might have been the equivalent of £10,000 to £20,000 in todays money, over 10 times less!

The population has changed massively, in 1948, it was 50M, and is now 60M, which in itself, with no other changes would be an increase in cost of 20% but now also includes a much higher percentage of the elderly who will cost more per-head to care for than younger people.

Certain treatments were not available in 1948 and are today and they are expensive! Cancer treatment varies between some low-level chemotherapy or basic surgery right through to modern cutting-edge drugs that might improve life-expectency by a few percent but at an enormous price. People might not like the idea that they are not worth the money but the organisation NICE has exactly the job of providing a value on human life for the NHS and deciding what is worth spending and what isn't.

Prescription drugs were very limited in 1948 but now, people get all kinds of things via the NHS. There is a fixed charge for some people but for many, they are free at point of use and these alone cost the UK was a staggering £10B in 2017. I don't have figures for 1948 but it would have been significantly less!

The culture of society has changed. With any "free" service, there is a danger that the privilege becomes expected and it is not only more common for people to visit hospital (why not when you meet a super qualified doctor who will examine your cut knee!) but even for some to get verbally agreesive if the "service" they receive is not as expected. I heard recently of someone getting physically agressive in a doctor surgery because they were unhappy that the government are stopping prescribing (i.e. paying for) medicines that are available over the counter. This puts obvious financial pressure on hospitals but also the fear of mistakes and litigation makes some Trusts either avoid providing a service at all, or they have to spend more time and money making sure they didn't miss something. Of course, that doesn't always work, people do make mistakes so the additional money might not be adding much value!

It's about money

All of these things wouldn't be a problem unless it was about money. Of course it is, there is no bottomless pit of cash to pay for things that people want, whether libraries or old people's homes. There are still people who think that either the government print whatever money they need and therefore under-investment means that they don't care, or otherwise that they have some massive bank account with a few trillion in that they should spend on the NHS to fix it.

There is another idea that somehow "I paid my tax", which means that it all goes into a pot and pays for my care. That is, of course, also not true. Your tax paid for the government spending the day after you paid it, it is not in a pot and it is not "owed" to you. With inflation, the total amount of money you paid in tax would cover nowhere near what you cost to the NHS, especially since it wasn't gaining interest in an account somewhere! The simple fact is that the NHS today is paid for by what people pay in tax today. As prices inflate, either wages (and tax) would need to increase consistently, spending has to be cut or the government has to borrow and hope that somewhere in the future we will do some kind of magic economy thing to be able to pay the money back.

The NHS is NOT free! It is free "at point-of-use" but unsurprisingly it is very expensive. Approx £111B per year. Roughly 18% of all government spending.

So this brings us nicely to the common accusation that "The NHS is really inefficient", it has "loads of middle managers" etc. and that simply some more efficiency would change everything. This unfortunately is a mantra for people who do not understand most corporate environments. Of course there will be inefficiencies and we will find areas where money can be saved today but these things don't sit still, you might save £10M today by buying power but then another contract you just signed will cost you another £10M next year, especially with so much contracting that is required for Trusts to meeting targets and quotas set by Politicians.

The Corporate world i.e. any organisation with more than 500 people and 4 levels of management (my definition) is rife with inefficiencies. There are jobs that we might not think are needed but you have to have 3 people producing graphs if you are required by law to prove that you are meeting targets. Could 2 people do the same job? Sure but good luck finding all these amazing people to staff an organisation that employs 1.5M people! There is a reality that some people are rubbish, some are amazing and most are average. If you only employ 5 people, you have the luxury of choosing those amazing people, more than 10 empoyees, it largely becomes impossible.

Could people work smarter etc. Again, yes in theory but is a nurse going to be allowed to use their intuition to make something better or are they made to tow the process line to make sure things don't go wrong? This is healthcare, there isn't much appetite for risk taking!

The simple maths is that the demand and expectation for health services is too high for the money that the government are willing to/able to provide.

What can we do?

There are a number of measures that the NHS can and does already take to help with the crisis of a bulging NHS problem but another simple truth that is lost on many critics is that there is no simple answer, otherwise it would have been done. Labour's answer seems to concentrate on more money but of course this can't really come from anywhere except taxes and most people are not prepared to pay more.

Taxation comes from various places, the main three are Income Tax, National Insurance and V.A.T. there are others like beer and fuel duty and inheritance taxes but over 90% comes from these three. The controversy is about who pays them and how much they could be put up.

National insurance is supposed to pay for the NHS but not everyone pays it. Of course, theoretically everyone pays it. In fact, when I was unemployed, I missed payments because I didn's sign on! The reality is that any state payments like pension and unemployment are basically higher so that the individual can pay their tax straight back to the government so in reality, only people earning are paying it (and their businesses), which leads to a difficulty. Should I pay more NI just because the elderly and possibly people who don't work and have even more chance of health problems and free prescriptions are costing a large part of the NHS cost? It's a matter of degree, of course, but that is the problem.

Likewise, income tax is paid by people who are working. The socialist ideal is, of course, that those who are productive take care of those who aren't but that assumes that those who are not working are still giving to the community in some way and that pensioners are not simply travelling around sightseeing and the unemployed are not at home watching Sky TV! Of course, these are not all the people but there are enough of them to make it a touchy subject for tax payers.

VAT is an intersting concept because it is based on the idea that those who can afford luxuries can afford to pay for stuff. Ignoring the few really random places where VAT is paid but shouldn't be (and vice-versa) it works OK except, again, it is like punishing the people who have done well in life to pay for those who cannot look after themselves. If we loaded VAT even more, we could raise money but VAT actually affects pretty much everybody because plenty of things we consider normal, like cake, are treated as luxuries for the purposes of VAT.

So raising money by raising taxes is a tool but it would be unpopular, not just generally but because of the idea that I pay more because I am doing better in life, not because I need more from the state.

So the flip side of making something work is to reduce the cost of the NHS. As previously mentioned, there would be scope for making some savings and various governments have already done some of this but keeping on top of it requires employing some more "middle managers" and things change as soon as a new drug or piece of equipment becomes available or something else comes out of patent and is made cheaper. The NHS have done well convincing Pharmacies to dispense generic drugs in place of named brands of the same thing.

The only big way to reduce the cost is to reduce the demand! This, again, is contraversial for the simple reason that not everybody needs the same thing. If you, for example, said that you could only visit the doctors a certain number of times before needing to pay, would that be fair to someone with a chronic illness? Would we be able to do it in a way that allows people with known genuine problems to visit multiple times and others to have a limit? You could charge people, again very unpopular because, "I pay my taxes" but the useful part here is that a) you could charge people who miss appointments i.e. they lose their payment and b) It provides a mechanism that makes people think, is my cut knee really worth £5 to see the doctor? Again, it creates problems for those with long term conditions and the elderly are potentially very vulnerable if they are living extremely frugally and are less likely to pay money to see a doctor. If you start exempting people then you fall back to the problem that only people who earn have to pay - which is unfair and doesn't solve the issue of reducing demand.

You could, in addition, reduce what treatments are available "for free" and have a system like many dentists and some hospitals. You get basic treatment for free but pay a bit more and you get more time and a private room or something like that. Of course, many people don't like, "two tier systems" "one for the rich and one for the poor" or "one for the 1% and one for everyone else" but unfortunately, there aren't many easy ways to distinguish people other than money or quota and since the main problem is lack of money, why not offer nicer food/services/beds/whatever for those who are willing to pay a bit more money. People could even get free upgrades!

 Conclusion

None of this is easy. Once an idea or service has become the norm, changing it in any way is hard and causes a backlash. Some backlash is expected, some is unreasonable but most people will probably agree that some changes need to be made.

The other difficulty is there will always be people who will lose more than others. In fact, the Labour Party frequently use the hackneyed expression, "The poor will be hit the hardest" which in most cases will always be true! If the country benefits but a small percentage loses out in some way (small percentage might still mean 100,000 people!) is it still worth it? Of course, the answer is always "yes and no". Yes financially and no, it will be a real problem for some people. Even with planning and time, changes will never be embraced with open arms and the population has to understand that we cannot afford the NHS in its current form (as well as many other services!).

The NHS, to their credit, have tried to encourage people to use services correctly - go to the pharmacy instead of your GP, go to your GP instead of A&E but there are no sanctions for people who ignore it. If I go to A&E anyway, I'm not going to be turned away (although I know in some more extreme cases you might) and the problem doesn't go away. If they could actually dictate what will and won't be looked at in A&E, that might help, also if they let more nurses carry out lower level diagnosis and treatment (again, I know this does sometimes happen) then we might reduce the need for a doctor looking at a cut hand.

Could we use technology more? I would personally be more than happy for a nurse in A&E to look something up on the computer to find out whether a wound needs referring/X-ray etc. or whether to simply bandage up and send me away.

Ultimately, only the people who work there can say what will work internally. Of course, more money and less people would help but that is not something a GP or hospital worker has much control over. There are plenty of well-meaning and able healthcare staff who could tell you in 5 minutes what the major problems are. Perhaps then you could list them all, estimate cost-benefit and knock a few over. If we scrap healthcare targets and gain £10M a year, is it worth it while at the same time losing some visibility over how different hospitals work etc.?

Monday 30 April 2018

Is GDPR really that bad?

Introduction

Sorry for the delay (I'm sure you were all upset!) but I feel I should be blogging more now that I have deleted my Facebook account and perhaps make some more useful comment and observations. I was prompted today by the news that streetlend.com was shutting down due to new GDPR regulations that are coming into effect in the EU at the end of May.

These have been a long time coming and basically formulate, more specifically, how personal data must be treated by any entity that is based in the EU or has any data from EU citizens in their system. The regulations were announced in 2012, first drafts coming in 2014/2015 and the final draft brought into law in January 2016, with just over 2 years for organisations to become compliant, after which, individual member states Data Protection Authorities can enforce action against individuals or companies who do not conform.

This is quite a long period of time but even large organisations like ICANN who are not based in the EU but process data from EU citizens have recently appeared on the radar for the simple reason that they have ignored the introduction of the regulations, have accepted more recently that they do need to do something and have then come up with the completely unprecedented idea that they can have a further year to sort things out! As far as anyone can see, you cannot have a moratorium on existing regulations (they have been law for 2 years now) and they are in the cross hairs for some large fines!

Streetlend Shutting Down

Anyway, I was reading comments about streetlend.com and why they shut down. Basically, the arguments on their site are that the regulations creates "uncertainty and risk that I can't justify taking". They also complain that GDPR creates the possibility of enormous fines, way above what most small companies could ever afford, that the requirements are ambiguous and that there are legal firms who are waiting to prey on small companies who might make a simple mistake and to end up causing a Court case that would be largely unaffordable for small companies and which therefore favours large companies who can effectively eat up the competition. The front page states that these regulations "add complexity and unintended side-effects for businesses within the EU".

Whether these views are genuine or whether they are hyperbole from someone who wants to make a point (anti-EU, anti-regulation, whatever) we will assume for the time being that they are genuine fears, but as the discussion shows, the arguments start becoming conflated and confusing because the reactions are about several different things. I want to look at these separately because, of course, any regulations have positives and negatives, for many companies, the obvious negative is "change" which usually comes at a cost, but ironically for this regulation, not necessarily!

Where did GDPR come from

Firstly, the background of the GDPR is a fleshing out of many existing data protection regulations that exist at national level. In fact, the EU version is based heavily on the existing UK Data Protection Act (DPA) and although it adds some more clauses, if you are already strongly in the spirit of the DPA, you might have very little additional work to perform to be compliant with GDPR. The most likely issue here is that the Data Protection Act, like many laws, appeared serious on paper but was rarely taken that seriously (at least in terms of enforcement), either by organisations or by the Information Commissioners Office (ICO) in the UK (the office who processes complaints about the DPA) and therefore, many companies didn't necessarily really understand or apply the spirit of the DPA, which is largely the same as the GDPR - What are you doing with data, why, how etc.

Now, this is very different from the USA, where many online companies are based and where privacy is based on a very fragmented and ad-hoc set of federal and state laws, which provide a very low bar in terms of privacy and data usage. For these organisations, they now need to apply GDPR level controls to their systems - some for the first time - and for smaller companies this is obviously a burden. What we have to remember here, is that the way that GDPR respects personal data is something that should have been happening anyway and if it was, then probably some additional wording on a privacy policy would have been enough, but for the grey companies who illicitly sell your data in a way that you probably wouldn't agree to, they should (rightly imho) have to become transparent. Recruiters, particularly, are very bad at keeping data for too long and 'accidentally' not removing you from lists when you ask for that!

Beware of the fines

People are very nervous about the new maximum fines which are 4% of turnover or 20 million Euros, whichever is greater. This certainly sounds scary but in the DPA, the maximum fine is only about £500K and the largest ever levied was actually £400K for Talk Talk. The reason for the larger maximum is simply that a company like Talk Talk can easily afford a few hundred thousand for a fine (The Chief Executive was paid £550K for 2 months work when she stepped down!). As the ICO have pointed out, the maximum fine is exactly that, a maximum. There is little point and no appetite for making an example out of small companies by bankrupting them with a large fine, unless their crime was deliberate, negligent, has a disproportional effect etc. In other words, the same as any other fine that is levied against a company. The fear that an innocent mistake would cause some large arm of the law to wipe you out is not only paranoia, it is hardly borne out in history. Clearly, if the company is already struggling, it is vulnerable anyway.

On the other hand, we should know that there is a large stick the ICO can use to beat companies with and there are many who have played fast and loose for too long. The likely outcome is that many of these will still evade action rather than the "good guys" all going out of business.

These favour the large companies

Almost without exception, new regulations are always easier to process by large companies because they already have legal teams. This is a reality of the capitalist system and nothing to do with the regulations. It is the same as saying that they "hit the poor the hardest". To balance this view though, larger companies also have much higher overheads, which means they have to either have very high income streams or they have to make higher profit margins on their sales to pay for these overheads. There are plenty of very large companies who have failed and either restructured (with lay offs) or have gone completely bankrupt so let us not fall into the habit of seeing the blessings of large companies without the curses!

A related note is this idea that somehow all large companies are looking to kill off the small companies so they can keep their market share. Of course, to an extent, it is likely that any company would rather not have competition, which makes their job easier, but for most large companies it is much easier to buy out a competitor than try and destroy them. Also, the GDPR doesn't add very much more than already existed to allow it to be used as a corporate weapon. You should be adhering to it regardless of whether someone has their crosshairs set on you.

Complexity

I liked one comment I read on Hacker News, which said that like a lot of Engineers, they had a natural dislike of regulations. We love the idea of the Victorian era where you built a bridge and it might collapse or it might not! We do not live there anymore: the danger we can present by abusing the trust that people put in us means we need to take this seriously and, like the DPA, the GDPR is not really complex at all! Read the principles here: principles and tell me that these principles are complicated?

"But the wording is vague!". Yes, that's always the case in regulations. If you over-prescribe the wording, then it doesn't fit into all the hundreds of thousands of companies that are trying to apply it. "What does lawfully mean?" You should know that for your business already, otherwise what are you doing?

In fact, despite all of the extra guidance, most of us could probably make a good stab at being compliant with these principles just from here. 

"Collected for specified, explicit and legitimate purposes" This sounds easy enough, tell them what you are doing and why!

The rest of the principles and these are explained (right to access, right to object etc.) are all fairly easy to understand, their reason is mostly pretty obvious and although some of them might be a pain (you cannot charge for access requests), they are not hard. In fact, they are much harder for larger companies who are likely to have a large number of systems processing data, perhaps many legacy systems that are not easily updated. So we can't really complain about the complexity.

Sticky Bits

There are some additional requirements which are more onerous, but they are still reasonable from the legal point-of-view even if they are a pain.

The one I find the trickiest is the right of portability: "The right to data portability allows individuals to obtain and reuse their personal data for their own purposes across different services." It is best understood with an example:

I sign up to a healthcare provider that takes all my medical data from hospitals and doctors and uses it to provide the service. I then decide that another company offers a better package but the first service has all my data and under the older regulations, that was tough luck! The new regulation is quite heavy on the idea that the first service must allow export in a "reasonable format" (presumably for certain industries this might already exist).

The idea is that it is your data so the service provider should not be allowed to silo it for their own use. Fair idea, understandable, but possibly a real pain for some organisations!

Another principle is that Data Processors are also legally liable for data handling, whereas previously, only the Data Controller is. For example, a company uses another company for storing Backups. The Backup company loses the data, which 'belongs' to the Data Controller, therefore, they are in the firing line. In the new regulations, the Processor (the Backup company) are liable for their own failings. Again, this is reasonable. I assume the Contractors I use are professional and following all the relevant regulations, how can I know if they are doing something dodgy?

Conclusion

Although the GDPR will create work for people who have not previously worked under a Data Protection Act, as regulations go, it is written in clear language that most professionals would understand without specific legal advice.

The idea that there is any risk that is disproportional to other new regulations is also groundless since the ICO have clearly said the higher end of the fines is for persistent and deliberate offenders. The workload for the ICO will be so large in the UK, that it is virtually a given that a small mistake in your system would warrant nothing more than a warning letter (if that).

There are weaknesses in the legal process and it does make legal experts and lawyers lots of money and that is a different problem to solve. If we needed an example to prove these problems in the legal world, however, the GDPR would not be one of them!