A year ago, I made a few predictions regarding IT in 2005. Let’s see how well I did.

• Didn’t come true at all: The home PC market will keeping declining and by the end of 2005, a new architecture will seriously threaten the PC for home web surfing, email and instant messaging. There will be serious talk of replacing the PC-on-every-desk model in many companies. Maybe Microsoft will be pushing an XBox2-Pro for business use.
• Came true, but Voice-over-IP is now everywhere, and we have things like Google Talk. So broadband is doing well:Videoconference-type broadband will still be out-of-reach for most home users and most small and medium businesses.
• Didn’t come true at all, but I have a 160 GB hard drive now: Whatever the big thing in IT is, it will have to do with storage. Job prospects in large cities will improve significantly for IT workers in 2005 and the growth will be driven by the new possibilities offered by infinite permanent storage. They will be significantly more data warehousing at the end of 2005 especially in smaller companies.
• Came true:Google will still be the most interesting Web company at the end of 2005. They will still be seen as a potent competitor for Microsoft.
• Came true: The Semantic Web will still be mostly at the same point it is now, at the end of 2005. That is, some nice ideas, including RDF and XML will stick around and find some uses, but OWL won’t take off.
• Came true partially, but new models for recommender systems didn’t arise: The Web will keep evolving. Personalisation will be a big thing: while the Web is now seen as a static graph on which people navigate, we will start seeing the Web as a graph around people. Social software will keep growing and growing in importance and won’t be based on ontologies or any such rigid model. New forms and models of recommender systems will emerge.
• Came true: Security will be a big thing in 2005 as it was in 2004, but we won’t make significant progress. People will install critically insecure software and they won’t care; or else, they will keep locking everything down.
• Came mostly true: eLearning in universities will keep on growing and we’ll have significantly more online courses offered by the end of 2005, though the push will come from students and deans, and not so much from Faculty members.
• Didn’t come true: eLearning outside universities may grow out of the PowerPoint or Flash models, but if so, only because some cool new technology, maybe based on XML, makes it possible.
• Didn’t quite come true, but is in the air: Year 2005 will be the year where the parallelization of systems and algorithms will become ubiquitous because of changes in CPUs.

So, my success rate is about 50%. I’m probably no better off than a random generator, but I’m a random generator with a personality.

Here are my predictions for year 2006…

• With Blu-Ray around the corner and about to invade many homes thanks to the PlayStation 3, 100 GB of storage on a single optical disk will be common by the end of 2006. Amazing video games using upward of 30 GB will come on the market and impress reviewers. I imagine a flight simulator containing the complete maps of the entire planet including every single house.
• Google will still be the most interesting player by the end of 2006. They will leverage their massive storage capacities to do amazing Data Mining and they will know, better than anyone else, what the pulse of the planet is. Google will start analysis social trends and will get into decision support.
• Generally speaking, year 2006 will be the year Data Mining becomes mainstream. Data warehousing will increasingly be a big deal for large corporations and we will see shortages in Data Engineering.
• Thanks in part to fancy open source content management software, eLearning will grow in most universities. By the end of 2006, we won’t be asking “why eLearning” but “how eLearning”.
• eCommerce will all be about personalization and Data Mining, and much less about work flow and web site design.

If you are a Python user, and you do Numerical Analysis, it might be time to move from Numerical Python to SciPy Core. I complained earlier about SciPy Core, but it seems that most of the problems I pointed out (missing inline documentation and broken functions) have either been fixed, or I wrongly pointed a finger.

I still have a few issues with the new package though:

• The naming convention for the LinearAlgebra package is awful. I don’t want to have to work with a package called “scipy.basic.linalg”: if you want to save space, don’t abbreviate LinearAlgebra to linalg and add three subpackages to it. Call the package scipy.LinearAlgebra, for example.
• There are glitches in the documentation. Both “help(scipy.basic.linalg)” or “help(scipy.linalg)” return a page describing “scipy.basic.linalg” as the “Lite version of scipy.linalg” and a list of 3 supported functions (Heigenvectors, eigenvectors, singular_value_decomposition). There are more than only 3 functions in this package!

On the positive side of things, you can now simply do “import scipy” to get the basic functions (like the scipy.array class). So, time to switch!

An early draft of Java Data Mining 2.0 (JSR-000247) is available.

In JDM 2.0, data mining includes the functional areas of classification, regression, attribute importance, clustering, association, feature extraction, time series, and anomaly detection. These are supported by such supervised and unsupervised learning algorithms as decision trees, neural networks, Naive Bayes, Support Vector Machine, K-Means, Apriori, Non-negative Matrix Factorization, and ARIMA.

JDM supports common data mining operations such as model build, test, and apply (score). JDM also supports the creation, persistence, access, and maintenance of metadata supporting mining activities.

Also in JDM 2.0, the standard includes extensions for basic text mining, statistics, and transformations integrated with the mining process. A particular implementation of this specification may not necessarily support all interfaces and services defined by JDM. However, JDM provides a mechanism for discovery of supported interfaces and capabilities.

Standard Deviations says he doesn’t get XSLT, RDF, XQuery, XLinq.

Here are some of his comments:

XSLT: It seems to me transformation is complex. It appears it requires a programming language. It strikes me I know a couple of those already. It strikes me that XSLT is yet another, and that I would rather use one I already use for everything else to do my transformation too.

I feel like there’s a big aha moment waiting for me in RDF, but it just doesn’t seem interesting enough to invest in getting there.

XQuery: The idea here is to have yet another XML based programming language? And this language is better than Python, Perl, or Ruby because?

My own answer is simple: XML is not, in any way, more powerful than Java, Python, Perl, Ruby or C#. XML is a standard way to write fancy documents (and by extension, fancy data structures) using a text stream. The key word is standard. XML is nice because it is widely supported. XML tools that are widely supported are nice to know. Period. There are no ah-ah moments waiting for you.

XSLT is nice enough because it is a powerful, yet friendly, declarative language. XSLT is also everywhere thanks, in part, to Firefox and Internet Explorer. So it is worth learning about XSLT because it is widely supported and there are respected XSLT standards.

The DOM API is also everywhere thanks, in part, to JavaScript and Java. Being available everywhere, and being usable, are really nice properties. The DOM API is also implemented in a consistent way. Yes, the DOM API is bad. It is verbose, it is hard to use. But then, so is C++ and we will still have C++ in 10 years.

I think that in 10 years, we will still have XSLT and we will still have the DOM API. Much of the rest will have gone away except for the truly good ideas because it is hard to get an idea widely supported. Supporting many alternatives is expensive.

My own advice is: don’t learn any XML technology unless you need it or unless it makes its way on your machine on its own. XSLT and the DOM API squarely fit this nice. XQuery, XML Schema, RDF and XLink don’t.

Wherever you are, have a good night!

I’m still looking for a Firefox extension that will turn any textarea (such as those found in forms) into a powerful text editor. Meanwhile, I just found Xinha Here! It turns any textarea into a pretty decent HTML editor. The little secret is that an HTML editor is also a text editor. Not a very powerful text editor, but you have search and replace!

In a day and age when I start wondering whether my kids will not learn about programming in a browser, a good browser-based text editor is really needed. We need more tools like Xinha.