Re: MySQL in the BI/DW space
Posted by: Peter Nolan
Date: May 23, 2005 03:58AM
Thank you for your response. I appreciate it very much...and the Cox case reads very impressively.
My background is I have been in IT for 23 years and in what we now call BI since 1991. Nowadays I spend most of my time implementing Sybase IWS and usually on Oracle though I have also done IWS on Sybase IQ.
My interest in MySQL.
We have recently released the 4th version of an ETL tool we use for building DWs. With this version we created a company to provide the ongoing support for that ETL tool. It supports all the 'usual suspects'. It relies on views and since 5.0 supports views and MySQL is getting some press in the BI space we thought we would try our tool on top of MySQL. So far, it seems to work....we are finding just one small problem which we are sure we will figure out.
What are we looking for in MySQL?
We were not really looking for anything specific at this stage because we were just testing to see if our ETL software would run on MySQL. Obviously MySQL is new to us.
Since we also 'open source' our software (at very low cost though not quite 'free') and MySQL is far less expensive to run than the major players we were considering supporting MySQL at the same level we support all the other databases. Because of the low cost it would seem to us to be a sensible option to support MySQL.
As part of us deciding if it worth us supporting MySQL I was interested to see what kind of headway MySQL was making in the DW space. If MySQL was making some headway it would be worth us supporting it.
We would not be interested in becoming MySQL experts so much as just making sure our software runs on it ok. That would provide the opportunity for people using MySQL to use our ETL software. (Obviously we think people using our software is a good idea.. ;-) )
I hope this makes sense for you...
One your question....
What do DW databases need?
Well, there is something of a list. You can always go down the list of oracle/DB2 options and you will get a good feel for what is needed.
For us, we always use dimensional models and so the number 1 need is for the optimiser to be star join aware and to delay reading the fact table until after the dimension tables have been processed.
The whole problem of providing cartesian product support in star joins. If this is not there then we need to fool the database into thinking it is.......as we used to.
The step up from that of course is the support of bit mapped indexes on keys for fact tables and vectored joins as is done in Oracle....however, this kind of thing is only required when the fact tables are large......SQL Server 2000 does not support such things so you can see a database can be quite usable in the lower range without it......
Of course, physical partitioning and optimiser awareness of physical partitioning is good in the higher end as well.....
As I said, the list of options that are useful are those that Oracle/IBM have put into their databases over the last 10 years. Easy to identfy and simulate if MySQL wants to go down that path...though I would expect it is rather a lot of development work and you would only want to do it if you were taking some databases away from Oracle/IBM/MSFT....:-)
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.