Big Data Consultancy


Harnessing the power of big data is a game changer, thanks to new technologies and much better open data support. Being able to use big data in a fast efficient way creates new commercial opportunities. Being able to use MapD (a graphics card powered database) for example, means being able to interactively play with huge quantities of data in real time, enabling analysts to spot patterns. Databases are getting equipped with AI, and anomaly detection (elastic for instance), providing another level of commercial intelligence 

It isn't all necessarily about huge quantities of data, either. Facets of big data and tools have paved the way for  data combining technologies, such as document based databases. A simple example: years of freely available weather combined with your own sales reports, maybe with the FTSE 100 history, may reveal some vital intelligence. More and more data is being provided online, years of Financial Ombudsman decision reports for example, can be analysed for commercial opportunities for a Compliance Advisor. 



Blurrt - Social Analytics

Since 2013 I've overseen the development of a social analytics platform. This has to ingest at scale large quantities of tweets mostly, and in real time provide reporting. Other social platforms can also be ingested as the platform has been built to be source agnostic, so Facebook, Twitch even CSV can be supported. This allows for analysis and overlay of different streams. 2 key challenges have been to 

a) develop a scaling architecture to continuously cope with data ingestion whilst being cost effective

b) develop an extremely fast reporting platform for real time analysis of the data

As of May 2017, we are able to track and report on (in real time) all UK election conversation, identify which party is doing best with what demographic, with metrics on 8 core emotions as well as sentiment, and much more.


S2 Partnership – Health and Safety

I built an integration into two leading statutory engineering systems (from Allianz and Zurich) from our commercial property management system. Combining inspection reports and risks from elevators, escalators etc. with general H&S statutory inspections and risks had never been done at that scale before in the UK. The unifying of these data sets provided some business challenges which were overcome and decreased the amount of work required, whilst also providing better visibility of risk across property portfolios regardless of source.


MIT – The challenges of tackling Big Data

In 2014 I successfully completed a distance learning course run by the Massachusetts Institute of Technology, partly as my current company was founded by members at an MIT conference. I have developed links with personnel at the Sloan School of Management and the Computer Science and Artificial Intelligence Laboratory, since attending a week long Industrial Liaison programme there.


Speaker at Digital 2014

For thw Wales Digital 2014 conference at the Celtic Manor I was asked to speak on a panel about how SMEs can leverage Big Data concepts within their organisation. Whilst my take was that the key component – huge quantities of unstructured data – was not relevant to most SMEs, certain facets of Big Data (e.g. the combining of multiple open data sources with the company's; weather data, sales data, FTSE etc) could be potentially useful. Moreover, I suggested that the huge progress in accessibility, in terms of data formats, making data publicly available, the development of tools to help analyse for anomalies etc. lowered the barrier for entry sufficiently for SMEs. They could therefore, cost effectively benefit from occasionally bringing in Big Data consultants, to help inform future business decisions.