Big Data's Challenge: Matching Business Needs With Technology

Since the earliest days of the financial markets, information has been a key element of success. In the past, market information was conveyed by methods that now seem quaint: carrier pigeons, personal conversations, printed materials sent through the post.

Today more data is generated in a 24-hour period than in entire centuries of the past, traveling at lightning speeds to all corners of the globe. Accessing, sorting, compiling, and leveraging that information is increasingly important in fast-paced markets and changing regulatory landscapes.

In this data economy, all kinds of businesses, from online retailers to pharmaceutical giants, are mining a wealth of information to better serve their customers, stay ahead of rivals, and improve the bottom line. The task is no less crucial in financial services.

[ Can Insurers Achieve Customer Engagement?. ]

The term “Big Data” has been used in a variety of ways, applied to everything from traditional relational databases to web-based sentiment-analysis tools. Just remember the three V's: the increasing velocity, volume, and variety of information available from a growing range of sources. All those bits and bytes only add up to something when they’re organized, arranged, and made coherent.

Not all analytics or data processing is big data. Trading and securities processing technologies have long been able to scale to meet the increased flow of electronic data resulting from market-structure change and increased electronic activity; high-speed trading is a good example. Complexity doesn’t necessarily mean big data. But big data is almost always complex -- which means it requires intelligent solutions if its potential is to be tapped.

[ Read the rest of this article on Wall Street & Technology. ]