Talking to myself - define the problem

Taking my own advise and talking to myself, I had a dialogue the other day and tried to hone in on step 1 of solving any problem - Definine the Problem. The subject was global corporate control of the onslaught of data and how to deal with it from an architectural standpoint. (I am a data architect by trade and nature.)

Thus:

(disclaimer: excuse me if it sounds like gibberish. it may be)

**** break snippet begins here ******


define the problem

all data is in a RDBMS - meaning its splayed out everywhere for efficiency and
speed of collection the tools for data manipulation is everywhere - the
knowledge, the Microsoft products (from Excel to SQL Server), open source, the
network itself, ubiquity of computers)people will never cease from digging,
turning, striving to understand data from unique perspectives

so what's the problem?

conflicting data numbers

is that so bad?

yes. numbers need to be definitive

even if there was an "official" set of numbers, anyone with "un-official" numbers can question it - causing churn, justification, backtracking, proving numbers, etc. And some people will be motivated by instinct to do just this.

a possible answer is the evolution of simplicity through complete
transparencythat means: instead of trying to hide complexity through middle-tier
software and access to raw data elimination, REVEAL the complexity (open source
code) and make it open and alterable and viewable by those that CAN and those
that CAN'T understand it.

What's this do?

It allows second-guessing to be done at the code level specifically
questioning
logic instead of the fruit of logic.

It eliminates churn driven second-guessing by those unable to understand
the complexity of the code itself.

but it doesn't dissolve the conflicting numbers issue... it only puts a handle on it at best.

another possible solution would be to allow the reporters of numbers to compete
and defend ala Capitalism or a Darwinian eco-system. the consumers of the data
will choose nourishing or diminishing the data providers into success or
extinction naturally - the best will survive and rise to the top.

that solution is naturally rejected because of the fear of the loss of control it produces.

this fear could possibly be mitigated by an "open-source" attitude management
(e.g., Wikipedia)

bottom line is that open information produces a certain amount of argument and
chaos (ala science itself)

Is it worth trying to eliminate this naturally propagated discussion with
Millions of $$ of new software, resources, and jobs?

..comes down again to a cost/benefit ratio.

That’s the final question.

Is it worth it? What do we get for what we pay for? (now the mystery of economy slips in and the circulation of the dollar itself.)

So again. What is the real problem? And What is the cost of that problem?

the problem is conflicting data.

the cost of that problem is churn and waste of money and time. but how much? can it be quantified?

difficult to do.

guesstimate twice the amount of people and time needed to do the job. (incidentally, the 2nd half of people that would be "cut" will not easily give up their purposes or jobs.)

What's the expense of fixing this assuming it can be fixed?

About 15 million $ for one middle-tier software package and 7 million $ for another middle-tier software package. Plus however many new people hired with invested time to implement, herald and rollout.

I'd guesstimate about the same about of people to implement, retrain, and hire
to do what jobs being eliminated. So in reality this is just a shift of skills
and jobs and peoples.

So if everything evens out, then the cost is
however much $ was sent out to software companies for product itself. (i.e. 15 +
7 = $22 million)

Will it solve the problem?

mostly not. because churn and questioning will still ensue due to the nature of
the people and the tools available and ubiquitous.

So cost/benefit ratio is $22M/$0

What about the benefit of the newer potentially robust views of software that will now be available that wasn't before, enabling better management.?

I'd have to say that the numbers were mostly already there before mid-tier
software and it will probably not be used more or less by management. ROI still
= $0.

So do we give up? (second question)

No.

What do we do?

Implement RAW DATA retrieval abilities across the board at a middle management
level through rollup into mart table pivots by an official group (Reporting
Services whether custom SQL, Business Objects, or Merced, etc.) so RAW data is
available back downwards to individual level generators (agents themselves,
their managers, and sites).

And use SAME RAW DATA rollups to report upwards to executive management
levels -

ALL WHILE KEEPING PULL CODE COMPLETELY ACCESSIBLE AND VIEWABLE.

Key points being that it must be RAW and TRANSPARENTLY GENERATED. (all 3 solutions tend to NOT be.)