The complexity of our society is the accumulation of many small decisions made by each individual
Some refer to it as the butterfly effect. The flaps of a butterfly lead to a tornado. In other words, many small perturbations would build up to a larger effect. This applies to many aspects. The financial markets, a nation’s identity, a community’s culture. There are all amalgamations of many small behaviours and decisions, made by you and me. We will present the use case of this via search analytics later.
Latent is the state of inertia before a material event happens. In latent combinations, these can lead to a market-moving event. To the informed, this is when intelligence can be applied, to preempt the big moves.
‘The stars are aligned tonight, hence X is likely to happen.’
‘This is a key resistance level. If it manages to break X, it could go all the way to Y’. (A practitioner’s favourite)
‘Something does not feel right, I better…’
‘There is something different about X today, could it be…’
‘This is some hot stuff, hence it could trend’ (For viral media)
These are what we experience in everyday life. We have this ‘gut’ feeling that something is likely to happen. Just that we can’t really ascertain it.
Machine Learning Approach
A right way to work this is to use machine learning. We brute force all the historical occurrences and the probability of what happened next. We then plug in our current state into the ‘learnt model’ and assess what is likely to happen next. A less fanciful to describe this will be a regression. For a real-life problem set, we may need more complexities to address the issues. Perhaps the correlation between the factors is non-linear? Or we need more repetitions to get an acceptable answer? Or we need to transform and change the form of the input data?
This is predictive analytics explained.
Using search analytics to uncover hidden intentions by individuals
When humans like you and I make decisions, we need to gather information before making a final decision. Information gathering in today’s context is very much ‘Googling’. With internet provision becoming more widespread, we turn to Google for an instant answer. For ‘data scientists’ like us who have the knowledge and means to get plugged into the backend side of the search engines, we are able to investigate what the combined herd is thinking at any point of time. This is a valuable avenue to uncover hidden intentions by individuals.
Google takes up 90.31% of the search engine market so plugging into Google API for search analytics will be sufficient. For context, frequency in keyword searches is shown to correlate with reports of flu infections in US, unemployment rates and US stock market trading volume. Intuitively, this should make sense. If we are falling ill due to catching the flu bug in the office, we will be searching for ‘Flu remedies’ on google. If our stock holding suddenly face a big decline in value, we will be rushing to find out more information on the stock.
Use case for an artificial plant retailer in Singapore, Urbangreens
This a generic use case example to demostrate the capability of search analytics. We use python and google trends API.
Urbangreens as a new plants retailer in Singapore. Consequently, many decisions need to be made:
- What items should we sell?
- Who should we sell to?
- What overseas market should we target?
- For SEO purposes, what are the keywords we should deploy?
- What complementary products should we sell?
- Who are our competitors?
Applying search analytics
Our first port of call is to check google search interest for ‘Artificial Plants’ over time. We feed in the data over time and visualize it.
There is a noticeable pick up in keyword trend post-2013 for both World and Singapore regions. Again, this can be due to the general pick up in general consumer sentiment. In the same way, this also correlated with recovering global GDP growth over the same period of time. Perhaps, this is an after-effect of the post-financial crisis, with the pick up of discretionary spending and on home decor.
For Singapore alone, search demand for ‘Artificial Plant’ has picked up in 2018. Could this be an interesting emerging trend in Singapore? Let’s watch.
Next, where should we export plants overseas?
We construct a heat map using a current normalized search interest across geographical regions. The data visualization is very clear. We should be looking to export to Australia, New Zeland markets for geographical proximity to Singapore.
Next, we need to consider our competitors, complementary products through correlated keywords analysis.
Uniquely, the related queries suggest that IKEA is a key competitor. We should also consider selling hanging plants, artificial flowers, aquarium plants correspondingly. These are related keywords a potential artificial plant buyer will look at.
Search analytics data could provide future insights into future trends of consumer behaviour. Combining large behavioural data sets could open up new insights into different stages of large-scale collective decision making.