In the latest developments, 16,000 positive Covid cases go missing ... because the results are transferred from the labs to Test and Trace via an Excel spreadsheet, which hit its maximum row count. Source.
Here's a question ... how many rows would each entry in the spreadsheet take? Obviously more than one, because how else could the spreadsheet hold more than a million rows of data? Though I suppose they could still be on Excel2003 or earlier with a 65536 row limit. Surely even the government IT doesn't still use Excel95 with a 16384 row limit?
I'm waiting for the clarification that they were actually using Google Sheets.
Apart from the issue of using a spreadsheet when they should have been using a database, Office software comes with a warning it should not be used for safety critical functions.
Don't think this counts as "safety critical" in that sense. "Safety critical" functions are the domain of provably-correct software, specially-expensive PLCs and the like. "Don't use this for safety-critical functions" is a standard disclaimer that comes with all ordinary commercial software. Basically it's a "we don't promise that a bug fix or upgrade won't break something" statement.
In the latest developments, 16,000 positive Covid cases go missing ... because the results are transferred from the labs to Test and Trace via an Excel spreadsheet, which hit its maximum row count. Source.
Here's a question ... how many rows would each entry in the spreadsheet take? Obviously more than one, because how else could the spreadsheet hold more than a million rows of data? Though I suppose they could still be on Excel2003 or earlier with a 65536 row limit. Surely even the government IT doesn't still use Excel95 with a 16384 row limit?
No it was because they exceeded 16000 columns - having laid out the data badly in the first place
"In this case, the Guardian understands, one lab had sent its daily test report to PHE in the form of a CSV file – the simplest possible database format, just a list of values separated by commas. That report was then loaded into Microsoft Excel, and the new tests at the bottom were added to the main database.
But while CSV files can be any size, Microsoft Excel files can only be 1,048,576 rows long. When a CSV file longer than that is opened, the bottom rows get cut off and are no longer displayed. That means that, once the lab had performed more than a million tests, it was only a matter of time before its reports failed to be read by PHE."
Wouldn't it be simpler to send test results for that day, rather than create massive files that need to be transferred around? What sort of bizarre system is this?
You must remember that they will not have invested in anyone extra to do data entry. There is probably some poor sod in middle management who has been told to pull and send the data from the labs systems by such and such a deadline once a week.
Wouldn't it be simpler to send test results for that day, rather than create massive files that need to be transferred around? What sort of bizarre system is this?
Wouldn't it be simpler to send test results for that day, rather than create massive files that need to be transferred around? What sort of bizarre system is this?
Part of a system that cost more than two aircraft carriers.
Wouldn't it be simpler to send test results for that day, rather than create massive files that need to be transferred around? What sort of bizarre system is this?
One invented by a team devoid of scientists of any description, I can only assume.
Wouldn't it be simpler to send test results for that day, rather than create massive files that need to be transferred around? What sort of bizarre system is this?
The middle manager deputed to integrate stuff together probably doesn’t know what a cvs file is. The middle stage is probably x numbers of sites sending stuff to be put together to then be sent to PHE, which is a separate agency. The database is probably after it gets to PHE or after it gets to Tracing.
What is the point of the 'Excel spreadsheet' stage???
Manager from outsourced consultant: "We've been told that the outputs of this system should be available for review at ministerial level"
IT specialist: "That's OK, we'll make sure the relevant people have access to the secure server holding the database"
MFOC: "But, that would mean the database would need to be accessible"
ITS: "Yes, and what's the problem?"
MFOC: "It'll mean everyone at ministerial level will need to be familiar with the software"
ITS: "It's a database, it's designed to make accessing the data easy ... I'll put in a couple of extra summary screens with graphs and stuff, easy"
MFOC: "But, we need it even more accessible ... couldn't you put it in Excel then the ministers can manipulate the data and produce graphs they like?"
ITS: *splutter* "What's the point? Without proper auditing of the data and pre-designed analytical methods that would be statistically meaningless."
MFOC: "We've been told that there really should be an Excel version"
ITS: "We've already decided that each lab send data in .csv format to avoid potential compatibility issues, they can read those files into Excel if they want to play politics with that data ... but I've just been offered a job designing a database for the German test data. Bye".
New IT Specialist: "OK, what's the state of the system?"
MFOC: "We'll, we've got all the data from the labs into Excel ..."
NITS: "Oh. Not ideal, but this system's needed urgent, so here's how you get the data from Excel into a database where it can be used ..."
Meanwhile in an office in Whitehall ... an incompetent government minister (is there any other kind these days?) is playing on his PC "So, if I divide this column by that, and plot the result against this unrelated column ... bingo! a pretty graph to show we have a world beating system and it's OK to go grouse shooting this weekend".
In the latest developments, 16,000 positive Covid cases go missing ... because the results are transferred from the labs to Test and Trace via an Excel spreadsheet, which hit its maximum row count. Source.
Here's a question ... how many rows would each entry in the spreadsheet take? Obviously more than one, because how else could the spreadsheet hold more than a million rows of data? Though I suppose they could still be on Excel2003 or earlier with a 65536 row limit. Surely even the government IT doesn't still use Excel95 with a 16384 row limit?
No it was because they exceeded 16000 columns - having laid out the data badly in the first place
The middle manager deputed to integrate stuff together probably doesn’t know what a cvs file is. The middle stage is probably x numbers of sites sending stuff to be put together to then be sent to PHE, which is a separate agency. The database is probably after it gets to PHE or after it gets to Tracing.
Thinking about it, and given that the problem is with one specific lab, I can see how it could happen ...
The PHE / Test and Trace database expects the daily input to consist solely of the most day's test results;
Almost all labs comply except AwkwardLabs LLP, which has configured its own database to send a list of all tests ever (in CSV format);
As everyone is under time pressure, rather than force AwkwardLabs to reconfigure its entire system, a minion is deputised to import the CSV into Excel, and then use Excel's filtering function to select the most recent tests, and pass them into the PHE database;
This system appears to work, so no-one fixes it, because all the IT specialists have more urgent things to fix;
A few days ago, the minion notices that AwkwardLabs' update is failing, and sends out a panicked email to their superiors, but since the Excel workaround is a side-of-the-desk job, the massively growing email struggles to find someone who understands CSVs, and, in the meantime, the figures have to get out every day ...
In the latest developments, 16,000 positive Covid cases go missing ... because the results are transferred from the labs to Test and Trace via an Excel spreadsheet, which hit its maximum row count. Source.
Here's a question ... how many rows would each entry in the spreadsheet take? Obviously more than one, because how else could the spreadsheet hold more than a million rows of data? Though I suppose they could still be on Excel2003 or earlier with a 65536 row limit. Surely even the government IT doesn't still use Excel95 with a 16384 row limit?
No it was because they exceeded 16000 columns - having laid out the data badly in the first place
What? Every entry was a new ... column?
According to Radio 4 this evening, it was in fact that the version of Excel they were using was so antiquated that they exceeded the row limit.
It appears from the bbc, the labs sent in cvs files, then PHE automatically combined them into an xls spreadsheet which it then sent on. (Ran a macro presumably.). It was the row rather than column limit that was violated though.
The reasons I still think this Serco’s fault, is because they are being paid billions *to design a system* that works. If Serco’s had said to PHE, combine it like this - use this file type - they would have done so.
It appears from the bbc, the labs sent in cvs files, then PHE automatically combined them into an xls spreadsheet which it then sent on. (Ran a macro presumably.). It was the row rather than column limit that was violated though.
The reasons I still think this Serco’s fault, is because they are being paid billions *to design a system* that works. If Serco’s had said to PHE, combine it like this - use this file type - they would have done so.
If Swerveco explicitly asked for the data as an Excel spreadsheet then it's hardly surprising that PHE does in fact use Excel ...
To be fair to both parties, there are better ways to combine CSV files than an Excel spreadsheet, but they would require someone to design and test them, and would then subsequently be a complete black box to the majority of users. Whereas the Excel solution can be set up quickly and at least with Excel you can sort of see what it's doing. The main nonsense is that someone had enough technical nous to programme a macro, but not enough nous to save it as XLSM instead of XLS.
Well, as someone obliged to use office365 - using excel - I think it defaults to xls.
ETA checked - it saves as xlsx
From memory, if you try to create a spreadsheet with a macro in it, Excel 2016 amuses itself by defaulting to XLSX and then telling you 'Sorry, you can't save a macro in that format'.
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
The software costs are not important. Much of it free. The costs are all in setting up and running it. Using Excel, if that's what they did is incompetent. Don't they have hardware and software people in-house?
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
I can point you at any number of mission-critical things that operate on a thin tissue of excel spreadsheets, because Dave knocked up some kind of proof-of-concept thing in excel, and then everyone sat round a table and said "can't we just use what Dave has".
What puzzles me is that the usual cobbled together drek that passes for databases in the public sector (*squints at SEEMiS*) at least doesn't lose information. This seems to be singularly awful.
What puzzles me is that the usual cobbled together drek that passes for databases in the public sector (*squints at SEEMiS*) at least doesn't lose information. This seems to be singularly awful.
Awful certainly. But having seen someone carefully throw away a whole month's worth of data because their hand-written data transfer scripts failed to check for a successful copy before deleting the original data (everything worked fine until they filled the target disk, then started silently failing), I don't think it's nearly bad enough to be singular.
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
The software costs are not important. Much of it free. The costs are all in setting up and running it. Using Excel, if that's what they did is incompetent. Don't they have hardware and software people in-house?
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to. And then later on, I doubt anyone is going to spend developers' time on reinventing a system that seems to work when there are so many other bits of the system that patently don't.
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
I can point you at any number of mission-critical things that operate on a thin tissue of excel spreadsheets, because Dave knocked up some kind of proof-of-concept thing in excel, and then everyone sat round a table and said "can't we just use what Dave has".
Hey! I resent that! I would have used Matlab (also probably the wrong tool, but still...)
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
The software costs are not important. Much of it free. The costs are all in setting up and running it. Using Excel, if that's what they did is incompetent. Don't they have hardware and software people in-house?
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to. And then later on, I doubt anyone is going to spend developers' time on reinventing a system that seems to work when there are so many other bits of the system that patently don't.
Worked real good.
Also suspect someone could have asked if any one anywhere in the world is dealing with a similar problem and situation and "hi there, what're you using, d'you mind if we chat?" they might have...
I suspect it was Excel 2010. Back when it was new it might well be configured to default to .xls for backward compatibility with Excel 2003. Office 2010 is widely used in NHS and successor organisations as that was where the EWA with MS left it in 2010. It's in the process of being replaced by O365 now, but whether that applies to no longer NHS bodies like these I have no idea.
From what I hear it wasn't file size or number of columns but exceeding the 65536 row limit. Using a column for each record would have been so batshit it would have made the decision to use Excel at all seem sensible.
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
The software costs are not important. Much of it free. The costs are all in setting up and running it. Using Excel, if that's what they did is incompetent. Don't they have hardware and software people in-house?
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to. And then later on, I doubt anyone is going to spend developers' time on reinventing a system that seems to work when there are so many other bits of the system that patently don't.
Worked real good.
Hence the words 'seems to'. I'm the one who brought this up on the thread in the first place, you don't need to tell me it didn't work.
My point is that when you have to set up a completely new and complicated system in a very short space of time, you're not going to get a theoretically optimum solution; you're going to get a lot of workarounds cobbled together with bodges.
Once again Matt has been let down by people not doing their job properly.
His job is to make sure they are doing their jobs properly.
You seriously expect him to personally supervise everyone ?
You don't seem to understand the nature of management. If you have the Top Job it is up to you to take an intelligent, engaged and receptive approach to those reporting to you - How are we going to do this? What resources are needed? Contractual obligations, deadlines, quality assurance - all the structure and mechanisms for delivering outcomes - you don't have to do it all, but you need to be aware that it should be done, and that someone is responsible for doing it.
This twat just throws money at it and says Bring me something I can puff in public, I don't care how, but make it quick.
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to.
Sure. But you could at least use Access rather than Excel. You know, and actual databasing program...
Once again Matt has been let down by people not doing their job properly.
His job is to make sure they are doing their jobs properly.
You seriously expect him to personally supervise everyone ?
There should be a chain of supervision, and the question is where does the buck stop? Personally, in this case I'd let Matt off the hook. I don't think there's any need for the buck to go that far up - Baroness Dido Harding would be the minister with appropriate responsibility for the system.
Though, the bigger picture issues would be at his feet, and possibly the PM and other Cabinet members. Why is there a botched together system put together at the last minute? The UK should have a test and trace system that's been developed over decades, rehearsed regularly and even called into action on a few occasions (this isn't the first pandemic respiratory disease in the last few decades, and at smaller scale we've had local outbreaks which would have stressed the system allowing lessons to be learnt and revisions to the system enacted). Oh, wait, we did have. We had a system based around local authority resources, the system that had been tried and tested through all those exercises and smaller real events (and, I should say that in those exercises the data handling is the easiest thing to test, you can get the participating groups to generate any amount of simulated data you need to pass through the system - so a fault with data input that relies on Excel, or any other weak link, would be far more likely to appear in these exercises than factors such as inability to procure PPE). But, the UK government decided to dump the tried and tested system* and replace it with an outsourced national system at the last minute, replacing the system that worked with a hurriedly developed system, and in that situation a botch or two is likely.
* It's possible that the testing labs in the existing system supplied data as .csv files with an Excel stage to the input (I don't know), but because that would be supplied to local authority public health services (who would then forward that to PHE) it would be very unlikely that the particular issue with the number of rows of data would be reached. It's also quite likely that each local PH service had their own unique solution to the how to take input data from multiple labs and prepare it for their own use and forward to PHE problem.
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to.
Sure. But you could at least use Access rather than Excel. You know, and actual databasing program...
If I understand the reports correctly (and indeed if those reports are correct), we're not doing anything particularly databasey at this point, we're just gluing CSV files together into a spreadsheet to be uploaded into the proper database. Although why the proper database requires a spreadsheet as an input, and/or why it can't amalgamate its own CSV files, are entirely valid questions ...
You'd have thought they would not use a spreadsheet at all and use a proper database program like MySQL.
I can point you at any number of mission-critical things that operate on a thin tissue of excel spreadsheets, because Dave knocked up some kind of proof-of-concept thing in excel, and then everyone sat round a table and said "can't we just use what Dave has".
Oh yes - and others built on even flimsier foundations. And when they break, they come running to us and ask us to build a new one. No money, of course, because the old one was fine really.
But not when something is being done from scratch. At great expense.
This is a system that cost £12Bn and is worth £12. So where did the money go? If not into building it properly?
Once again Matt has been let down by people not doing their job properly.
Oh poor Mr Hanock. The biggest project he - or most other ministers - will probably ever be involved in, and he ommitted to be involved at a basic level to be sure that it was done right.
Once again Matt has been let down by people not doing their job properly.
His job is to make sure they are doing their jobs properly.
You seriously expect him to personally supervise everyone ?
Don't you think he should be supervising the supervisors? If not why not? If that's not his job what do you think his job is? Who is at the end of the line management?
Once again Matt has been let down by people not doing their job properly.
"Once again"
There's a pattern here. That pattern is Matt Handcock making terrible decisions about who gets the jobs. After the first few times, any reasonable person would conclude that it's Handcock that's the problem.
Once again Matt has been let down by people not doing their job properly.
It's more probable that some people have been given a very difficult job to do by their superiors without the appropriate support, resources and funding. In a way, I would actually include Hancock in that. Whatever it is he may be 'good' at, he's clearly not effective - or not being allowed to be effective - in his current role. Either way the responsibility for his appointment - and the chosen process of track and trace - lies with government leadership.
This is an issue of ensuring effective oversight over a technical, complex, life-saving process. Not just a case of delegating someone the job of cutting the onions for a hamburger, and finding out too late they were using a blunt knife.
Once again Matt has been let down by people not doing their job properly.
His job is to make sure they are doing their jobs properly.
You seriously expect him to personally supervise everyone ?
You don't seem to understand the nature of management. If you have the Top Job it is up to you to take an intelligent, engaged and receptive approach to those reporting to you - How are we going to do this? What resources are needed? Contractual obligations, deadlines, quality assurance - all the structure and mechanisms for delivering outcomes - you don't have to do it all, but you need to be aware that it should be done, and that someone is responsible for doing it.
This twat just throws money at it and says Bring me something I can puff in public, I don't care how, but make it quick.
I certainly do understand the nature of ,management and I do know that if junior management messes up, they take the blame and the person at the top is in the clear.
Take the Metropolitan Police for examle, an organisation far smaller than the NHS. If an Inspector messes up and does not do their job properly,the Commisioner is not criticised.
Once again Matt has been let down by people not doing their job properly.
His job is to make sure they are doing their jobs properly.
You seriously expect him to personally supervise everyone ?
You don't seem to understand the nature of management. If you have the Top Job it is up to you to take an intelligent, engaged and receptive approach to those reporting to you - How are we going to do this? What resources are needed? Contractual obligations, deadlines, quality assurance - all the structure and mechanisms for delivering outcomes - you don't have to do it all, but you need to be aware that it should be done, and that someone is responsible for doing it.
This twat just throws money at it and says Bring me something I can puff in public, I don't care how, but make it quick.
I certainly do understand the nature of ,management and I do know that if junior management messes up, they take the blame and the person at the top is in the clear.
Take the Metropolitan Police for examle, an organisation far smaller than the NHS. If an Inspector messes up and does not do their job properly,the Commisioner is not criticised.
However Hancock is minister. It doesn’t matter who does the wrong thing. It is different from business. That said how many times have we seem chairmen retire because of the mess a company is in. It is not just one issue either it is a catalogue.
Take the Metropolitan Police for examle, an organisation far smaller than the NHS. If an Inspector messes up and does not do their job properly,the Commisioner is not criticised.
If a single Inspector makes a mistake, the Commissioner isn't criticized. If many Inspectors make the same mistakes, then a good working assumption is that the fault lies not so much in the Inspectors, as somewhere up the chain of management.
In this case, it's not "a single Inspector" making a mistake. Those responsible for creating the track and trace system failed to ensure data integrity. That's a mistake at the level of the manager responsible for the whole track and trace system.
Take the Metropolitan Police for examle, an organisation far smaller than the NHS. If an Inspector messes up and does not do their job properly,the Commisioner is not criticised.
If a single Inspector makes a mistake, the Commissioner isn't criticized. If many Inspectors make the same mistakes, then a good working assumption is that the fault lies not so much in the Inspectors, as somewhere up the chain of management.
In this case, it's not "a single Inspector" making a mistake. Those responsible for creating the track and trace system failed to ensure data integrity. That's a mistake at the level of the manager responsible for the whole track and trace system.
Comments
I'm waiting for the clarification that they were actually using Google Sheets.
Don't think this counts as "safety critical" in that sense. "Safety critical" functions are the domain of provably-correct software, specially-expensive PLCs and the like. "Don't use this for safety-critical functions" is a standard disclaimer that comes with all ordinary commercial software. Basically it's a "we don't promise that a bug fix or upgrade won't break something" statement.
There's a counter claim in the Guardian that it was down to rows and numbers of tests:
Apply your own discount factor.
Never underestimate the power of stupidity.
One invented by a team devoid of scientists of any description, I can only assume.
Well, quite. Amateur hour or what?
Lab systems > CSV file > Excel spreadsheet > Database > PHE / Test and Trace
What is the point of the 'Excel spreadsheet' stage??? A database should be able to read a CSV file without Microsoft's assistance.
IT specialist: "That's OK, we'll make sure the relevant people have access to the secure server holding the database"
MFOC: "But, that would mean the database would need to be accessible"
ITS: "Yes, and what's the problem?"
MFOC: "It'll mean everyone at ministerial level will need to be familiar with the software"
ITS: "It's a database, it's designed to make accessing the data easy ... I'll put in a couple of extra summary screens with graphs and stuff, easy"
MFOC: "But, we need it even more accessible ... couldn't you put it in Excel then the ministers can manipulate the data and produce graphs they like?"
ITS: *splutter* "What's the point? Without proper auditing of the data and pre-designed analytical methods that would be statistically meaningless."
MFOC: "We've been told that there really should be an Excel version"
ITS: "We've already decided that each lab send data in .csv format to avoid potential compatibility issues, they can read those files into Excel if they want to play politics with that data ... but I've just been offered a job designing a database for the German test data. Bye".
New IT Specialist: "OK, what's the state of the system?"
MFOC: "We'll, we've got all the data from the labs into Excel ..."
NITS: "Oh. Not ideal, but this system's needed urgent, so here's how you get the data from Excel into a database where it can be used ..."
Meanwhile in an office in Whitehall ... an incompetent government minister (is there any other kind these days?) is playing on his PC "So, if I divide this column by that, and plot the result against this unrelated column ... bingo! a pretty graph to show we have a world beating system and it's OK to go grouse shooting this weekend".
Thinking about it, and given that the problem is with one specific lab, I can see how it could happen ...
According to Radio 4 this evening, it was in fact that the version of Excel they were using was so antiquated that they exceeded the row limit.
The reasons I still think this Serco’s fault, is because they are being paid billions *to design a system* that works. If Serco’s had said to PHE, combine it like this - use this file type - they would have done so.
If Swerveco explicitly asked for the data as an Excel spreadsheet then it's hardly surprising that PHE does in fact use Excel ...
To be fair to both parties, there are better ways to combine CSV files than an Excel spreadsheet, but they would require someone to design and test them, and would then subsequently be a complete black box to the majority of users. Whereas the Excel solution can be set up quickly and at least with Excel you can sort of see what it's doing. The main nonsense is that someone had enough technical nous to programme a macro, but not enough nous to save it as XLSM instead of XLS.
ETA checked - it saves as xlsx
From memory, if you try to create a spreadsheet with a macro in it, Excel 2016 amuses itself by defaulting to XLSX and then telling you 'Sorry, you can't save a macro in that format'.
The software costs are not important. Much of it free. The costs are all in setting up and running it. Using Excel, if that's what they did is incompetent. Don't they have hardware and software people in-house?
Our trust has, I think, just bought it back in house - but only just, it was Serco for a long while.
I can point you at any number of mission-critical things that operate on a thin tissue of excel spreadsheets, because Dave knocked up some kind of proof-of-concept thing in excel, and then everyone sat round a table and said "can't we just use what Dave has".
Awful certainly. But having seen someone carefully throw away a whole month's worth of data because their hand-written data transfer scripts failed to check for a successful copy before deleting the original data (everything worked fine until they filled the target disk, then started silently failing), I don't think it's nearly bad enough to be singular.
Unfortunately.
Yebbut Excel works more-or-less straight out of the box. When you're under pressure to get something set up, you're not going to waste time setting up MySQL to amalgamate CSV files if you don't have to. And then later on, I doubt anyone is going to spend developers' time on reinventing a system that seems to work when there are so many other bits of the system that patently don't.
Hey! I resent that! I would have used Matlab (also probably the wrong tool, but still...)
Worked real good.
Also suspect someone could have asked if any one anywhere in the world is dealing with a similar problem and situation and "hi there, what're you using, d'you mind if we chat?" they might have...
From what I hear it wasn't file size or number of columns but exceeding the 65536 row limit. Using a column for each record would have been so batshit it would have made the decision to use Excel at all seem sensible.
You seriously expect him to personally supervise everyone ?
Hence the words 'seems to'. I'm the one who brought this up on the thread in the first place, you don't need to tell me it didn't work.
My point is that when you have to set up a completely new and complicated system in a very short space of time, you're not going to get a theoretically optimum solution; you're going to get a lot of workarounds cobbled together with bodges.
No, we expect him to appoint people to run things who at least don't already have a track record of losing data.
You don't seem to understand the nature of management. If you have the Top Job it is up to you to take an intelligent, engaged and receptive approach to those reporting to you - How are we going to do this? What resources are needed? Contractual obligations, deadlines, quality assurance - all the structure and mechanisms for delivering outcomes - you don't have to do it all, but you need to be aware that it should be done, and that someone is responsible for doing it.
This twat just throws money at it and says Bring me something I can puff in public, I don't care how, but make it quick.
Sure. But you could at least use Access rather than Excel. You know, and actual databasing program...
Though, the bigger picture issues would be at his feet, and possibly the PM and other Cabinet members. Why is there a botched together system put together at the last minute? The UK should have a test and trace system that's been developed over decades, rehearsed regularly and even called into action on a few occasions (this isn't the first pandemic respiratory disease in the last few decades, and at smaller scale we've had local outbreaks which would have stressed the system allowing lessons to be learnt and revisions to the system enacted). Oh, wait, we did have. We had a system based around local authority resources, the system that had been tried and tested through all those exercises and smaller real events (and, I should say that in those exercises the data handling is the easiest thing to test, you can get the participating groups to generate any amount of simulated data you need to pass through the system - so a fault with data input that relies on Excel, or any other weak link, would be far more likely to appear in these exercises than factors such as inability to procure PPE). But, the UK government decided to dump the tried and tested system* and replace it with an outsourced national system at the last minute, replacing the system that worked with a hurriedly developed system, and in that situation a botch or two is likely.
* It's possible that the testing labs in the existing system supplied data as .csv files with an Excel stage to the input (I don't know), but because that would be supplied to local authority public health services (who would then forward that to PHE) it would be very unlikely that the particular issue with the number of rows of data would be reached. It's also quite likely that each local PH service had their own unique solution to the how to take input data from multiple labs and prepare it for their own use and forward to PHE problem.
If I understand the reports correctly (and indeed if those reports are correct), we're not doing anything particularly databasey at this point, we're just gluing CSV files together into a spreadsheet to be uploaded into the proper database. Although why the proper database requires a spreadsheet as an input, and/or why it can't amalgamate its own CSV files, are entirely valid questions ...
Oh yes - and others built on even flimsier foundations. And when they break, they come running to us and ask us to build a new one. No money, of course, because the old one was fine really.
But not when something is being done from scratch. At great expense.
This is a system that cost £12Bn and is worth £12. So where did the money go? If not into building it properly?
Oh poor Mr Hanock. The biggest project he - or most other ministers - will probably ever be involved in, and he ommitted to be involved at a basic level to be sure that it was done right.
Lazy turd who will rot in hell. Along with Dido.
Once again, you miss the point. As others have troubled to explain.
"Once again"
There's a pattern here. That pattern is Matt Handcock making terrible decisions about who gets the jobs. After the first few times, any reasonable person would conclude that it's Handcock that's the problem.
It's more probable that some people have been given a very difficult job to do by their superiors without the appropriate support, resources and funding. In a way, I would actually include Hancock in that. Whatever it is he may be 'good' at, he's clearly not effective - or not being allowed to be effective - in his current role. Either way the responsibility for his appointment - and the chosen process of track and trace - lies with government leadership.
This is an issue of ensuring effective oversight over a technical, complex, life-saving process. Not just a case of delegating someone the job of cutting the onions for a hamburger, and finding out too late they were using a blunt knife.
That is a very very good question.
But...there are no pockets in a shroud...
I certainly do understand the nature of ,management and I do know that if junior management messes up, they take the blame and the person at the top is in the clear.
Take the Metropolitan Police for examle, an organisation far smaller than the NHS. If an Inspector messes up and does not do their job properly,the Commisioner is not criticised.
However Hancock is minister. It doesn’t matter who does the wrong thing. It is different from business. That said how many times have we seem chairmen retire because of the mess a company is in. It is not just one issue either it is a catalogue.
If a single Inspector makes a mistake, the Commissioner isn't criticized. If many Inspectors make the same mistakes, then a good working assumption is that the fault lies not so much in the Inspectors, as somewhere up the chain of management.
In this case, it's not "a single Inspector" making a mistake. Those responsible for creating the track and trace system failed to ensure data integrity. That's a mistake at the level of the manager responsible for the whole track and trace system.
I agree with the BIB