If you believe that, then I have a bridge to sell you
The media and blogosphere have been consumed by the troubled rollout of the Healthcare.gov health exchange website, with a long list of management and technology failures adding fuel to the political fires. The latest finger pointing in this seemingly endless saga lays blame on the selection of MarkLogic's NoSQL database as a key factor in the site's ongoing problems.
According to GigaOm, "...the issue seems to be that it's harder to find database admins and other techies that know NoSQL databases inside and out whereas there's a ton of existing expertise in SQL databases from Oracle, Microsoft and IBM."
The NY Times put it even more bluntly: "Another sore point was the Medicare agency's decision to use database software, from a company called MarkLogic, that managed the data differently from systems by companies like IBM, Microsoft and Oracle."
The implication here is that the site's problems would have somehow been avoided, if only a more traditional database had been used instead.
While I have great respect for both of these publications, does anyone really believe a better solution to a project involving many disparate sources of information, complex logic, and a dynamic interface, which must be built in a very short timeframe would have been to select IBM, Microsoft or Oracle? The idea that legacy mega-vendors have the agility required for a project of this scope is absurd, as the states of Oregon, Pennsylvania and the US Air Force have all recently learned the hard way.
Let's take a look at the real issues at play here.
Selecting a NoSQL database like MarkLogic, or more precisely in this case, an XML database, means that all of the Healthcare.gov data sources would have to be converted to XML. Of course that's a monumental task, but it's no more difficult and time consuming than the arduous extract, transform and load (ETL) processes required by traditional relational databases because of their fixed schema. The enormous time and cost associated with ETL is precisely why new technologies are emerging.
The only real advantage the legacy vendors hold over XML technology is that most every developer already knows SQL, versus the XQuery interface being used with XML, which is known for its steep learning curve. But there are newer, more agile options available, which fully support SQL, but don't require the months or sometimes even years of manual effort it takes to complete complex legacy ETL processes.
The bottom line is that choosing the "same old, same old" legacy relational database technology is by no means the safest path. Success can only be realized by deeply analyzing the particular problem you are trying to solve, understanding the nature of all the information that is needed to solve it, evaluating the skills of the people involved, and exploring newer, more innovative approaches that might offer a more direct path. Then – and only then – can you be sure to select the technology that best meets your requirements.