Date
21 November 2018
As the public becomes increasingly aware of privacy and the risk of data leakage, the role of data middle office is evolving. Photo: Freepik
As the public becomes increasingly aware of privacy and the risk of data leakage, the role of data middle office is evolving. Photo: Freepik

Challenges of extracting value from data

I’ve been among the first who put forward the concept of data middle-office. The idea came from my experience at Alibaba, when I was assigned to Taobao to help tackle its problem of having a fragmented data system.

When I arrived at the job, I soon discovered three issues.

First, different data analysis teams varied a lot in their capability. Second, the data department was constantly overwhelmed with all kinds of requests. Third, business departments, which the data department served, were not impressed with the latter’s findings. Quite a few data analysts resigned as they felt their inputs were not valued.

Different departments were blaming each other, and the morale was fairly low. It’s probably hard to imagine that five years ago the tech giant encountered considerable difficulties.

After in-depth research, I deduced four major challenges when operating a data department.

1. There was an enormous amount of data on a daily basis, and it’s difficult to identify the most useful information.
2. Data was scattered around in different departments and there was a lack of efficient consolidation
3. It’s often too late when business abnormality was spotted through data analysis.
4. Different analysts had their own approaches

In the end, we reached a consensus that someone should introduce changes in a systematic and technological way.

Since this department has to liaise with business departments on one hand and a number of data technology units on the other, calling it a middle office became a natural choice.

We discovered that a number of key issues had to be resolved through cross department coordination. That includes setting the standards for data collection, data sharing among different departments, data security guarantee before and after sharing, data quality assurance, etc.

Instilling in everyone the mindset of “create together and use together” was not an easy job.

It was once believed that centralization of data was the best approach in maximizing the value. But as the public becomes increasingly aware of privacy and the risk of data leakage, the role of middle office is also evolving.

In the future, as the Internet of Things development speeds up, dealing with a huge amount of widely dispersed data would again redefine the role of middle office.

This article appeared in the Hong Kong Economic Journal on Nov 7

Translation by Julie Zhu

[Chinese version 中文版]

– Contact us at [email protected]

RC

Venture Partner of Sequoia Capital China, former head of the data committee and vice president at Alibaba Group.

EJI Weekly Newsletter

Please click here to unsubscribe