News from the PDC 2005: XQuery, LINQ and friends, SQL Server 2005 September CTP
First, we have released the September SQL Server 2005 CTP for download. This is the last CTP before the final release. Yeah!
I will write my customary change blog on what has changed in the XML area a bit later this week.
This year's PDC again is quite exciting. First my presentation on XQuery in SQL Server 2005 (slides) was very well received, second there is some cool stuff being presented - which I knew of before, but seeing it publicly announced and seeing the audience getting excited is still cool and exciting, and third - and most importantly - I am getting some high-quality interactions with our customers. If you are at PDC and want to talk with me, feel free to come by the DAT track lounge in the great hall and ask for me (I will be there Thursday for much of the day).
My code samples that I used during my demo are now up on the COMMNET session download page. I will also place them on my blog for those that want to see the code.
There is lots of cool stuff being presented, starting with Tuesday's keynote. Lots of cool UI and data organization stuff in Vista and Office got demo'ed. Even though some of it is available in other OSes, it is probably the first consumer OS that we ship that I personally am looking forward to. Organizing data according to queries and new visualizations of the data organization in nice 3D is nice! And while we ship the sidebar some time after the MacOS, I have used a research version of it at least two years before Apple ship theirs...
While they did not show any of the cool plumbing features of Office 12 (like the XML support), there were some UI features (like auto-preview of formatting when hovering over a formatting option) that just shows that innovation is still possible.
The new Indigo (sorry, Windows Communication Foundation) stuff is cool, especially the protocol agnostic aspects. Roger Wolter and Gerald Hinson even demo'ed a prototype that used WCF to communicate over the SQL Server 2005 Service Broker!
For me the most interesting presentations are the LINQ related talks. LINQ is a new query infrastructure in the CLR that allows you to write more declarative, query like expressions over your CLR objects (LINQ), relational data (DLINQ) and XML (XLINQ). It builds on the CLR 2.0 features such as nullable types, delegates, IEnumerable etc.. Anders' PDC 2005 presentation TLN307 gives a great overview how LINQ builds on top of these.
While not being directly involved in the work, I was reviewing it and providing feedback to Anders, Luca and Dave along the way (aka "being a pain" :-)). As a former APL- and LISP-programmer, seeing the ability to program more declaratively on lists and sets, and seeing lambda expressions and lazy evaluation makes me very excited. The ability to write code more declaratively will allow programmers to become more productive and will allow us to more effectively optimize these declarative expressions. Obviously, as database programmers, we know all about the benefits of declarative programming! :-)
Now LINQ still has some challenges (besides being a tech preview that will have to do lots of work to get completely integrated): First the full aspects of the CLR is still procedural. So any optimization will be more complex than optimizing SQL (or even SQL inside T-SQL). The ease of programming can mislead people to start performing selections and joins in client main memory on large amounts of data that they fetch from the database. Since this not only will transport large amounts of data and stress the memory usage, it also will not have access to indexing and cost information that could help LINQ based optimizer. So there is lots of interesting work ahead to improve this :-). Although note: a DLINQ query that operates only on relational data will be shipped to the database server and has a way to execute a stored proc and map the result to objects!
Now let me give you my PERSONAL take on the relationship between XLINQ and XQuery (and DLINQ and SQL):
XLINQ (DLINQ) is basically providing XQuery-like (SQL-like) functionality on top of LINQ. It provides a functional and more declarative way to create and query XML documents (relational data) than for example the DOM (XLink uses about 30% less memory than the DOM). Since it is build on the language independent framework and operates in the CLR type model, it obviously makes sense to not try to retrofit SQL or XQuery onto that model (since it would provide a common syntax with a non-common semantics). Whether the LINQ family will replace SQL or XQuery in the database is questionable, due to some of the more side-effecting semantics of some of the operations (e.g., the reparenting of elements in XLINQ) and the obvious benefits of cross-industry standardization of SQL and XQuery.
I however find it a much better solution than trying to make XQuery a new general purpose programming language. As I said during a panel at the recent SIGMOD XIME-P workshop in Baltimore: XQuery will succeed by influencing the next generation of programming languages, regardless of whether it will become one itself or not!
As to whether XLINQ is making XQuery on the client-tier superfluous - this needs to be seen. I think here it boils down to a benefit-cost analysis of user-benefits and -costs and resource requirements to implement both. I am certainly interested in hearing your feedback either here in the comment section or by contacting me directly!
One question that was asked during Anders' overview was (paraphrased) on whether LINQ will become the query language of choice in the database. Anders' answer: We are thinking about it. My comment: Yes, we are thinking about it and about some other things in this space. However, it is too early to say anything about when and how exactly. I would like to make it clear however, that we are committed to continue to support SQL and XQuery in the database.
Here are some interesting linqs :-) for more on LINQ and friends:
It is surely an exciting time to work on programming and query languages and systems!