Modern data management, the hidden brain of AI
The current state of the art in deep learning comes from a brilliant idea to model algorithms based on how the human brain works . The structure of mathematical calculations is inspired by the way neurons connect. Why not? Computers are becoming more human-like with video, audio and vibration sensors.
The name AI implies that it is a replica of human intelligence in silicon form. It’s easy to forget the hidden brain that makes AI useful. Let’s use neuroscience as a metaphor for understanding this concept.
The typical adult brain weighs about 3 lbs. and consumes 20 WATTs of power. It is a remarkable machine. This efficiency-seeking function is what Daniel Kahneman, a Nobel-prize-winning psychologist, refers to when he describes System 2 thinking .. He demonstrated that our subconscious, and therefore low-powered, way of processing information is possible. It works more often than the executive function, which is more powerful.
Neuro-anatomy experts believe memories are encoded in emotions, but that emotions are not stored individually. They are basically references that are stored in the limbic. The basic idea is that we can recall an event and then look up the feelings about it. It can also have a profound influence on how we subconsciously choose our actions.
This limbic system is located in the middle of the brain and influences future decisions. It uses emotional memory to help us make better or worse decisions. We make poor choices when we lack context for risk and reward.
In the same way, AI analysis without the right data can lead to a flawed future. It is important to discuss how “all the right information” should be organized and presented. AI should consider the management of messy, high-volume, and unstructured data as important as the limbic system to the predictive function of a human brain.
But, there are other factors that can affect automatic decision-making. Let’s explore the brain metaphor more. Kevin Simler and Robin Hanson argue in their book, The Elephant in the Brain: Hidden Motives in Everyday Life, how unconscious we are about the nature of our own behaviors. They argue that we behave according to social motivations in a manner similar to our primate “cousins”. It doesn’t matter if you think this is evolutionary biology or learned in our family of origin. What matters is understanding that there is another hidden part to our human brains.
This blind spot may also explain why technologists often view data management as a phenomenon of culture. Usually, pundits only focus on data management in two dimensions. The first is technology-focused. It starts with byte sizes, throughput and access patterns. This platform mindset allows for the procurement, storage, as well as availability of data. It is strongly biased to metadata (data about the data), as this is the driving wheel. The second dimension most commonly used is process. This system-level view encompasses the entire pipeline from acquisition at the source to sorting and shuffling to cataloging to presenting and finally to archive. It is the farm to table point of view. Or, rather, farm to Tupperware point. Technology is a perspective that focuses on the “how”. The third dimension, and perhaps the most invisible, is culture. It refers to a set of behaviors that are anchored in a shared belief system and bound together by group norms. Culture is the pulley string of technology and process. It is the most overlooked aspect of data management.
Many institutions rush to implement technology and tool processes without understanding their culture. They would be better served to model themselves in the way that positive psychologists study the most successful people. These researchers study the beliefs and behaviors that are common among the most successful people.
While it would be valuable to present some case studies to support this point, we will only present a summary of the findings of those who are the most successful in data management. It all starts with a shift of belief systems about data. Data is no longer an artifact of events. It’s an asset with enormous economic implications. It can appreciate in value over time, unlike other items on your balance sheet. With that in mind, here’s a list of new behaviors associated to a shift in mindset about data.
- Data is federated into a fabric, not centralized nor siloed.
- Knowledge is organized by context and tagged by both publishers and subscribers.
- Models are preserved for continuous learning and accountability.
- Transparency (observability) mitigates legal and regulatory pressures.
- A broader view of ethics expands beyond the initial concerns for privacy.
- Machine learning automates data engineering tasks.
- Knowledge workers become value-creation workers.
- Top-down, data-driven decisions evolve into bottom-up shared insights.
- Data is measured in economic terms and not accounting terms.
So, if your organization aims at exploiting AI, do not overlook the importance of modern data management and the fundamentals that make it up. Start by comparing the current state to the desired one. To overcome the gaps, you need to use a cross-disciplinary approach. In formulating a gameplan, you should rely heavily on technologists as well as process engineers, economists, and organizational developers.
If you have interest in going deeper into this modern data management philosophy, please check out this white paper authored by Bill Schmarzo, the dean of big data, and an esteemed colleague at Dell Technologies.
This content was produced by Dell Technologies. It was not written for MIT Technology Review.
I’m a journalist who specializes in investigative reporting and writing. I have written for the New York Times and other publications.