End-user Computing (EUC) and in particular spreadsheet risk, has been well documented recently with one of the most high-profile headlines concerning the test and trace system for COVID-19 in the UK. Other examples include issues where formulas were implemented incorrectly within a spreadsheet, in one example leading to billions of dollars in loss to JP Morgan due to the miscalculation of their VAR position. However, what is the other side of the coin? What is the lost opportunity of having your data and processes tied up inside many versions of a tool where there is no reconciliation, data quality checks, and no mastering of information? Industry is moving rapidly towards a data and analytics arms race, where the winners and losers are being decided by the ability of organisations to leverage their time and energy efficiently and effectively. This is enabled by advanced analytics that enables firms to:
To get to the front or even the middle of the pack, then you need to be sure of your data. You also need to have your data to hand and available to the analysts, and data scientists to run their models. It is still a fact that 80% of a data scientists’ job is taken up with the consolidation, cleansing and conforming of data and training sets prior to being able to deploy their models. Given the escalating salaries of these resources, it is extremely expensive to have them performing work outside of their paid for skillset. If the real view of data is hidden inside spreadsheets, then it may not even be possible to pull together a valid data set without months of preparation work.
It is not uncommon for hard working users, in answer to business questions, to have formed a complex web of EUCs and manual processes over dozens of years to resolve business challenges. In organisations where there has been heavy M&A activity, it is not uncommon to find that data and business rules tied up in EUCs are the only place where there is truth in the organisation. However, more than likely the truth of one EUC can quite often contradict a second EUC that is reporting or analysing a different business problem. This leads to inconsistencies and eventually a lack of confidence in the results, which means that organisations are not able to capitalise on their information and insight.
One example that comes to mind where an analyst had diligently and very carefully organised a set of EUCs by date over several years. The total number of snapshots saved was around 90 and represented the reported truth for several key metrics that the business used. One request came in that required a trend of the KPIs to support business decisions. The task required the analyst to open each of the spreadsheets in turn, conform the data across all 90 instances and then extract the key metrics. The effort involved was around 3 weeks work. In this example, by the time the work had completed the business opportunity had passed. What isn’t clear is, could this knowledge have prevented a disaster or enabled the organisation to take advantage of an opportunity?
It is clear from the striking headlines that there are risks in having your data and processes in ungoverned EUC processes, as well as potential for regulatory fines. However, it is also as important for people to be aware of the limitations that are caused by a large EUC estate in the modern fast-moving world.