Recommendation engines haven’t always enjoyed the best rep. Back in the day you’d head online to buy your young niece a doll and then spend eternity battling ads for pink stuff and Hello Kitty.

For us customers, personalisation once meant websites presenting our own behaviour back to us with all the charm and sophistication of predictive text. With everyone stuck on the same merry-go-round, customers and brands gained precious little.

But that was then. In 2006 a mathematician called Clive Humbry coined the phrase “data is the new oil”, and sure enough, within a decade it was lubricating all cogs of digital progress. Today the constant analysis of millions of users via billions of data points – along with next gen data science has made the online world, and beyond, a radically different place.

Most companies with an eye on the future can see the value and potential in deploying advanced data yet many smaller brands and organisations can’t see beyond the complexities and expense. However, the bar to entry is getting lower by the day.

It no longer requires a Silicon Valley budget to get a forensic understanding of users through their own psychographic, sentiment and aspiration data, or to put those insights to use.

Sure, cynicism often accompanies the big data debate. Reducing people to data can sound cold and callous – and it’s not like collecting data has the best reputation right now. But it’s worth remembering that big data innovations are man-made things. Machines might do the heavy lifting but algorithms represent the latest iteration of human problem solving. Progress. It’s a tradition started by our monkey ancestors.

To see progressive data-innovation in the modern world we needn’t look further than Spotify. When the streaming giant realised that listeners had largely given up on music discovery, Spotify bods spliced psychographic and behavioural data with individual listener patterns. From there, Spotify found a way to offer fresh music recommendations to users with near pinpoint accuracy.

Through instant feedback loops the project quickly found its stride, and the more Spotify’s 70 million users engaged (which they did in droves), the more developers were able to tweak and sharpen the algorithm, thus the product, and increase its effectiveness.

Spotify identified an opportunity to plug a gap: by understanding customers via data, they were able to reintroduce the masses to music discovery. In this case the product was music but the principles apply cross-sector.

Some brands and businesses may believe their data capabilities or infrastructures are insufficient for algorithm-based innovation. They may feel they haven’t the purse strings or the know-how to compete in the big data age. But as the data game has matured, an array of off-the-shelf products are within reach of companies big and small.

Amazon Web Services, Microsoft’s Distributed Machine Learning Toolkit, Kaggle and IBM’s Watson Developer Cloud are among the open-access tools and platforms even skeptical amateurs can use to explore the possibilities.

Google Cloud’s AutoML is a drag and drop interface whose ultimate aim is to democratise AI and countless start-ups comprise the AIaaS industry, providing Artificial Intelligence as a Service so firms can explore new worlds of options.

Like in the Spotify example, when a hyper-personalised data solution flows from brand to customer, everybody wins. When companies can harness user data and produce something of benefit to people’s lives then trust flows back upstream. Symptoms of trust are retention, engagement, loyalty, brand advocacy and a healthier bottom line. Who’d say no to all that?

Perhaps the data/ oil quip has expired. One’s much more accessible than the other – its possibilities are endless.