I’ve been among the first who put forward the concept of data middle-office. The idea came from my experience at Alibaba, when I was assigned to Taobao to help tackle its problem of having a fragmented data system.
When I arrived at the job, I soon discovered three issues.
First, different data analysis teams varied a lot in their capability. Second, the data department was constantly overwhelmed with all kinds of requests. Third, business departments, which the data department served, were not impressed with the latter’s findings. Quite a few data analysts resigned as they felt their inputs were not valued.
Different departments were blaming each other, and the morale was fairly low. It’s probably hard to imagine that five years ago the tech giant encountered considerable difficulties.
After in-depth research, I deduced four major challenges when operating a data department.
1. There was an enormous amount of data on a daily basis, and it’s difficult to identify the most useful information.2. Data was scattered around in different departments and there was a lack of efficient consolidation3. It’s often too late when business abnormality was spotted through data analysis.4. Different analysts had their own approaches
In the end, we reached a consensus that someone should introduce changes in a systematic and technological way.
Since this department has to liaise with business departments on one hand and a number of data technology units on the other, calling it a middle office became a natural choice.
We discovered that a number of key issues had to be resolved through cross department coordination. That includes setting the standards for data collection, data sharing among different departments, data security guarantee before and after sharing, data quality assurance, etc.
Instilling in everyone the mindset of “create together and use together” was not an easy job.
It was once believed that centralization of data was the best approach in maximizing the value. But as the public becomes increasingly aware of privacy and the risk of data leakage, the role of middle office is also evolving.
In the future, as the Internet of Things development speeds up, dealing with a huge amount of widely dispersed data would again redefine the role of middle office.
This article appeared in the Hong Kong Economic Journal on Nov 7
Translation by Julie Zhu
[Chinese version 中文版]
– Contact us at [email protected]