These days I am reading the new book of Stephen Few, Signal. As always with books of Stephen Few, it sets my daily work in a new perspective and is inspiration for several ideas. One of the chapters I particularly like is about ‘correlation and causation’.
To summarize the conclusion: it is important to always use your head!
This is a conclusion I can really relate to. Nowadays with new technologies and new social media datasources there is more opportunity to find correlations between variables.
But what to do with these correlations?
Just using these correlations without knowing if there is a meaningful relation is not sound advice. As Stephen Few explains, finding a correlation between two variables is a potential step on the path of finding a cause, which can lead to something beneficial.
Also, the number of meaningful relations in the world will not increase by the number of correlations we find. It will only make them harder to find. In that sense, Big Data will give us a lot of additional correlations, but will this help in finding meaningful relations that matter?
As explained by Stephen Few, for every observed relationship between two variables X and Y, we can only conclude that X causes Y when:
- It is certain that there is a real relationship (ruling out unreliable data measurements, statistically significant, the sample size is big enough).
- Y does not cause X.
- It is not the case that something else, related to X, causes Y.
Ruling out these three points is something you need your head for. And tools like SAP Design Studio and SAP Lumira can help you there with visualizations and analysis capabilities.
To conclude: always use your head!