29 Sept 2009

Cycle of Acquiring Knowledge

I have been doing some reading into data mining recently. It's really interesting stuff.
To explain it, lets take an example:
Lets say you work in a business that has been working for some years. They have their own way of running themselves and their processes. One day someone comes along and suggests that the business should automate some of their procedures by developing a website that would interact with customers and process their orders.
Some nice PHP/MySQL guy or girl is hired and develops the website and the cycle of acquiring knowledge has unwittingly started.

The diagram below describes how management starts to process its own business knowledge and passes them down the chain so they can be implemented into the application.
Eventually, every transaction is saved in the database. Over time, the database has more and more data and then the management of the company asks to get reports from this data.

Why would they ask that? Because they are interested to get an overview about the operations from this new part of the business.

What they could also benefit from is finding patterns and trends from the data which can help the business increase its own business knowledge. Meaning, data mining.

The diagram below explains the process for that:
Here the data goes upstream and back to the business's management. Assuming they find trends or patterns that help them be a better business and more competitive, they should apply that knowledge into the operations of the business.

Now, I have a question....

Lets say, that the business decided to take the NEW business knowledge and apply it to their website. For example, every person who buys a red hat would also probably buy a green tie (like what 'amazon recommends' does ) or if you come to this landing page, there is an 87.2% chance you are only interested in these lines of products and nothing else.
So they would ask the developers to add that new rule to the website which would in turn make the website better to its users (hopefully).

This means that the cycle went from down to up (like in the first diagram) to up to down (like in the second diagram) and back again. Ergo, a cycle.
And my question is.... how fast on average, do companies repeat the cycle?
How often do they learn from their data lying in their databases and apply it to their businesses?

I'm going to guess around 6-12 months for a whole cycle, but its a wild guess based on no data whatsoever and in the data mining book I'm reading, you shouldn't really guess with no data.


  1. Hi!

    Q: "how fast on average, do companies repeat the cycle?"

    A: "it depends."

    It is completely dependent upon the dynamics of that particular line of business. That said, I don't think many companies will do it as a matter of principle. From what I have seen, larger companies will do absolutely nothing as long as business is perceived as "good enough".

    Another thing to consider is the period of time that it takes before a change in the organization or change in a process actually has any effect on performance. Again, this will depend upon the type of change.

    BTW - just curious...what data mining book are you reading?

  2. Hi Roland

    The book I am reading is this: