Running Hundreds Of Evolution Users

Over the last few months, we have had an increase in number of concurrent Evolution users, and also have had an increase in calendar and email usage. Evolution makes heavy use of cache when interacting with GroupWise and we were starting to have disk IO performance problems from 8am until about 10:00am. This is the heaviest load, when everyone is downloading heavily from the post office. The disk drives were getting so busy that the UI was starting to slow and performance was not acceptable. Interestingly, CPU usage was under 10% even with 250 people.

We had just ordered our annual allotment of servers. We use a trickle approach and move servers around based on changing requirements. Three new HP DL580G5s arrived . The older DL580s used 15K 3.5" Ultra320 SCSI 72GB drives. The new server uses 10K 2.5" SAS 146GB drives. Below are the specs, the difference in speed is amazing. We also had previously used RAID 5 which might have contributed to the bottleneck. We configured the new server and used RAID 1+0. A backup and restore of /home and IP number change the new server immediately went live with only a 15 minute interruption. The new server is so fast that when you hit [ Send/Receive] the dialog flashes and disappears so quickly it can barely be seen.

Disk performance is excellent even with 250 people:

iostat -x 10

Linux 2.6.16.54-0.2.3-bigsmp (oa3) 12/20/07
avg-cpu: %user %nice %system %iowait %steal %idle
0.97 0.00 0.24 0.01 0.00 98.81

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 0.00 71.90 0.00 22.30 0.00 1507.20 0.00 753.60 67.59 0.01 0.43 0.25 0.56

Previously, the drives were %util from 75-100%

Technical Specifications:

• Hard Drive Capacity : 72.8GB
• Generation: Ultra320
• Data Transfer Rate: 320 MB/sec
• Rotational Speed: 15,000 rpm
• Form Factor (Drive): 3.5-inch low profile
• Interface: Wide Ultra 320 SCSI; LVD
• Data Storage Device Type: SAS (Server Attached Storage) device
• Hard Drive Device Type: Hard drive for server/storage unit (Hot-plug)
• Height: 1 inch
• Pin Configuration: 80 pin Hot Swappable/ Pluggable
• Hotswap Tray: Included (Attached)

Technical Specifications:

• Hard Drive Capacity : 146GB
• Generation: SAS
• External Data Transfer Rate: 3.0 GB/sec
• Rotational Speed: 10,000 rpm
• Form Factor (Drive): 2.5-inch low profile
• Interface: SAS (Serial Attatched SCSI)
• Data Storage Device Type: SAS (Server Attached Storage) device
• Hard Drive Device Type: Hard drive for server/storage unit (Hot-plug)
• Height: 0.591"
• Width: 2.75"
• Hotswap Tray: Included (Attached)

Largo Desktop Video

I have recorded my first video to show everyone the new Compiz-Fusion desktop on our thin clients. The speeds that you are see are under normal loads, and exactly what the users experience. I'm just about finished setting up all of the icons and software launch scripts.

Notes:
- The blinking is not seen on the thin clients, movement is smooth
- I demonstrate the MIME bars I wrote to make it easier for them to process files
- Evolution is being moved to a new server next week that is much faster, you are seeing the speed of the current machine. Preview pane display is *much* faster on the new server.
- Video encoding for the web is new to me, so I'll do better next time. :)

Bliss

Bliss is your first 4-way Quad Core server. :)

DBera Is My Beagle Hero

We use Beagle extensively at Largo for searching historical documents. We have hundreds of thousands of documents going back 20 years. Each night I build a static index of each department, a static index of the city-wide shared directory and then the users personal directories. One of the requirements we have is to block users from searching departments of which they are not a member. We also had the requirement that we wanted users to be able to search individual static sources as well, to reduce the number of hits. The only way previously to do this was to have a zenity front end run before Beagle, and display all available indexes available to each user and then allow them to toggle the desired sources. It then passed this list to the beagle daemon and started the UI. Ok, it worked. But if you wanted to change your criteria you had to close Beagle and relaunch. Not very friendly.

So DBera cooked up a small fix that allows you to select from the sources via the UI itself. When a user runs Beagle now it figures out which sources they have access to search. It then starts the daemon with all of these sources and displays a dialog as below:



The UI then comes up and you are able to alter your search criteria on the fly. While all sources appear in the pulldown, only those connected with the beagle daemon will return results.



It's working perfectly for us. There is an ongoing conversation about how to integrate something like this into the main build. The patch right now is hard coded for our needs, but it sure seems like this would be helpful for other people. If you have comments or suggestions, add them to the feature request.

DBera, thanks for this step forward.

Friday Afternoons

Sometimes you do fun/stupid things on a Friday afternoon, especially after a week of troubleshooting and debugging.

I can now confirm that Stella works fine with Compiz-Fusion and Pulse Audio via remote display to thin clients. :) [shot below]

Have a great weekend all.