We’re starting to see a deluge of companies who businesses are all about making data analysis/science/insight “easy for the non-expert”. We’ve been here before, quite a few times sadly. When I started writing software 12 years ago, there was great excitement in the air – finally we could use tools to design software, then press a button that would create our whole beautiful design in code! Then we could just hire some barely-sentient code monkeys to fill in the ‘easy bits’ like method definitions and those pesky database access routines.
It was a disaster. The fundamental problem was that by the time you’d crafted your beloved design and polished it to a high shine, the world had moved on. What may have worked on day 1 of the project was now hopelessly inadequate. We should always remember the maxim “no plan survives contact with the enemy”, the enemy here being the shifting reality of what your software needs to deliver.
Another major problem with this approach was the proliferation of so-called Software Architects, beings of such insight and experience that they didn’t even need to code anymore! Since they didn’t code, they couldn’t experience the grinding pain of trying to jam their grandiose designs into a reality-shaped hole.
Fast-forward to today – data is big, Data Science is even bigger (as a buzzword anyway), and we’re all short of the right people. The answer, however, is not to make tools that hide the complex, ever-shifting reality of the analytical process. It’s to make people better at doing this stuff. And there’ll be no magic off-the-shelf solution that can achieve this, any more than giving a terrible golfer great clubs will make them win The Masters.