e-Literate (Michael Feldstein) posted recently on the need for creating a community and framework for evaluating both open source and proprietary LMSs. I hope something comes of it and am willing to lend my support to the effort, as long as it is in the tradition of open source. We need a shared and open methodology for comparing learning management systems.
Edutools provides a functionality comparison matrix but that's only one-dimension of the problem. How do we evaluate the scalability, performance or inter-operability claims of vendors or open source projects? What does the total cost of ownership for these systems look like? One can always hire consultants. But what do consultants know?
A neutral framework and objectively evaluated benchmarks would be very useful.