2009 Evaluation Team Report: Web Conferencing Evaluation
Direct test sessions
The team conducted test sessions with 4 products already being used at Penn, and for which we already had active accounts. They were Adobe Connect Professional, GoToMeeting, eLluminate, and WebEx. The test sessions were done without participation from the vendors; each was led by a member of the evaluation team.
The products were each evaluated according to the following criteria:
In these test sessions, the team:
- Attempted to randomize sample (asking people to connect with various platforms, over assorted connection conditions including on and off campus wireless connections).
- Collected data about each participant, their machine, and conducted a bandwidth test in association with each evaluation.
- Use the same evaluation form was for each product with lots of space for free form comments and response. The evaluation form provided with an automated way to collect and compile data and also a script for demonstrating each product.
The team found that Adobe Connect and GoToMeeting performed best. Adobe Connect got the highest score in 6 of the 11 test categories, including Chat, Polling, File Trasnfer, Integrated Audio Performance, Video Performance, and Ease of Joining Session. GoToMeeting got the highest scores for Screen Sharing, Latency and Overall Impression.
RFI to Vendors
The team worked with the Penn Purchasing department to prepare a Request for Information (RFI) to be sent to several webconferencing vendors. The vendors contacted were: Adobe Connect, WebEx (various flavors), GoToMeeting, Lotus Sametime, Microsoft Livemeeting, Yugma, Dimdim, Genesys/Intercall, Centra Emeeting (Saba), iLinc, Alcatel/Lucent, ATT Connect, Dialcom, and Netviewer
Each vendor was asked to provide the following:
- Company Background
- OS and Browser
- Privacy and Security
Responses were received from Connect Solutions (AdobeConnect), Dimdim Enterprise, Genesys/Intercall (WebEx), iLinc, Saba Centra Emeeting, Citrix (GoToMeeting), and WebEx corporate
- Adobe Connect and GoToMeeting were the best performers in the test sessions.
- GoToMeeting is simple & easy to use; it's well suited to research groups, business meetings or other activities which just basic features (audio, screen sharing, file exchange). GoToMeeting is also a good choice to include in mission continuity planning for instructional activities.
- Connect Pro is more powerful, but with a steeper learning curve; it is well suited to planned distance learning activities which can effectively leverage the advanced feature set
- iLinc looked pretty good based on vendor response. Would be worth a look for those who are not happy with Connect Pro or GoToMeeting
- DimDim: some key features lacking
- Saba: no support for Vista
- Since Connect and GoToMeeting scored highest in the tests and are already being used successfully at Penn, the team recommends these products to other groups who are looking for webconferencing solutions. Specifically, we will
- Work with Purchasing to explore pricing options for Adobe Connect and GoToMeeting
- Conduct SPIA assessments for Connect and GoToMeeting
- Arrange demo for iLinc (if anyone is interested)
- Coordinate with Mission Continuity planning groups to see if GoToMeeting should be included as part of contingency plans for teaching
The work of the evaluation team focused on products that met our working definition of web conferencing tools. However, there are a variety of other tools, many of them free, which can be used to support collaboration.
- Adobe Connect Now is part of the acrobat.com suite. It provide a very complete set of online collaboration tools, including free meetings with audio,video or screen sharing for up to 3 participants
- eLuminate Vroom is a free version of their webconferencing product. The free version includes the full feature set, but is limited to 3 participants
- Skype allows free audio-only online conference calls for up to 25 participants. It now also includes screensharing and file exchange capabilities for 2-party calls
- Yugma offers a free version of their collaboration product. The full feature set is available for 15 days; after that users have access to a reduced feature set in the free version.
- Dimdim free version includes most features of their "Pro" version, for groups of up to 20.
- Instant messaging clients including iChat, GoogleTalk, Windows Live Messenger and others provide file sharing, video support and (in some cases) screen sharing.
When using any of these free services, users need to be mindful of good security practices and the need to protect confidential information.
- Most of these products provide encryption for data in transit, and that's a good start. But users often don't know if any information or files transmitted during online sessions are cached by the hosting service, or otherwise vulnerable to unintended disclosure.
- Identity verification on these systems tends to be weak and simply trust participants to honestly self-identify when joining the session.
- In general, you should should NOT use any of these services to exchange confidential information including medical, academic or employment records unless the service has been carefully evaluated for information security and privacy practices.