Thursday, July 24, 2008

#Performance

I am starting to get some numbers for those of you interested in GNOME performance, and for those that are considering deployments. Our GNOME server is up to 106 concurrent users. More and more people are coming over each day, and it's interesting to watch the server scale. We have two identical machines, each able to run the entire City. Our total user load should be around 300 concurrent when finished, split between the two servers.

Raw data:
- Looks like it took around 8GB to get 106 users logged in + OS
- Looks like we could get about 300-400 total on this machine without swapping.
- The parent dbus-daemon is the top process 99% of the time.
- The python process for avant-window-navigator is a steady chew on the server.
- 17 users disabled gnome-panel and enabled avant-window-navigator
- Total of all CPUs sits around 10-15% consistently.
- Server is 4 CPU, dual core, 3.66Ghz
- 74 of 106 logins enabled Compiz and 3D effects.
- 15 total users are using NX from low bandwidth remote sites (no 3D for them).
- I can't add .desktop files with this user load, it hammers the server too much




Anyway, things are running well and looking forward to project completion in 30 days.

11 comments:

Pádraig Brady said...

I'd love to see the output from ps_mem.py on this machine.

Anonymous said...

Would be nice to get the avast and dbus people to focus on fixing those a few bugs...

Mark said...

Hi,

I'm a developer for the Awn/Awn Extras project. The dock itself doesn't use a persistent Python process - what you're seeing is most likely some applet that has a memory leak. The one that I personally know of is the digitalClock applet, which I fixed in trunk.

We would be more than willing to help you address your problems with Avant Window Navigator (as an aside, why do people [obviously not yourself] have trouble spelling it?) and the applets. Particularly, we need to know exactly which applet is chewing on the server. We have a relatively active community of both developers and users at our IRC channel, #awn on irc.freenode.net, which is probably the best place to get support at this time.

Thanks for adding Awn as an alternative to the panel in your deployment!
-malept

Dave Richards said...

Pádraig Brady: I generated ps_mem and the multi-user type stuff is below:

raw sausage: I don't know if these are bugs as much as things that just consume resources when lots of people are online. I'm learning as I go; we have never had this deployed in such big numbers.

mark: Will hit you guys up soon, yes the digital clock is on the panel for all users. I force in ApplicationBrowser + Digital Clock + Logout on the panel to simulate the basic features gnome-panel

Here is ps_mem:
10.6 MiB + 2.3 MiB = 12.9 MiB nxssh (29)
12.8 MiB + 776.0 KiB = 13.6 MiB mapping-daemon (115)
14.3 MiB + 820.0 KiB = 15.1 MiB gnome-keyring-daemon (116)
13.0 MiB + 2.5 MiB = 15.6 MiB sshd (59)
16.6 MiB + 2.1 MiB = 18.7 MiB rdesktop (9)
17.5 MiB + 1.4 MiB = 18.9 MiB sh_gnome_xplane (6)
23.2 MiB + 1.3 MiB = 24.5 MiB sh_gnome_bar_pi (8)
26.1 MiB + 1.3 MiB = 27.4 MiB sh_rdp03 (9)
22.5 MiB + 6.0 MiB = 28.5 MiB awn-applet-activation (16)
28.7 MiB + 772.0 KiB = 29.5 MiB dbus-daemon (119)
22.9 MiB + 8.4 MiB = 31.3 MiB bar_pictures.py (5)
23.6 MiB + 13.1 MiB = 36.7 MiB mono (4)
43.0 MiB + 452.0 KiB = 43.4 MiB ssh-agent (116)
43.6 MiB + 1.3 MiB = 45.0 MiB sh_gnome_avant (15)
38.3 MiB + 10.8 MiB = 49.1 MiB eog (3)
40.9 MiB + 10.4 MiB = 51.3 MiB avant-window-navigator (15)
43.2 MiB + 11.4 MiB = 54.6 MiB nxnode (30)
57.4 MiB + 3.3 MiB = 60.8 MiB wfica (15)
71.4 MiB + 3.1 MiB = 74.5 MiB gnome-vfs-daemon (118)
64.1 MiB + 11.4 MiB = 75.5 MiB gdm (103)
79.8 MiB + 2.3 MiB = 82.1 MiB bonobo-activation-server (117)
72.7 MiB + 17.5 MiB = 90.2 MiB nxserver (29)
94.6 MiB + 6.7 MiB = 101.4 MiB metacity (43)
129.5 MiB + 4.1 MiB = 133.6 MiB gnome-screensaver (114)
145.3 MiB + 13.2 MiB = 158.6 MiB gnome-terminal.real (24)
165.6 MiB + 7.3 MiB = 172.9 MiB emerald (73)
243.2 MiB + 8.0 MiB = 251.2 MiB gnome-settings-daemon (115)
270.7 MiB + 9.4 MiB = 280.0 MiB mail-notification (115)
272.9 MiB + 9.4 MiB = 282.3 MiB gnome-session (116)
320.4 MiB + 11.5 MiB = 331.9 MiB main-menu (110)
454.0 MiB + 16.0 MiB = 469.9 MiB gnome-panel (116)
465.1 MiB + 11.2 MiB = 476.3 MiB python2.5 (42)
571.0 MiB + 4.3 MiB = 575.3 MiB nxagent (15)
632.4 MiB + 20.2 MiB = 652.6 MiB nautilus (115)
652.5 MiB + 12.8 MiB = 665.3 MiB application-browser (116)
738.0 MiB + 1.8 MiB = 739.8 MiB gconfd-2 (117)
920.7 MiB + 4.8 MiB = 925.4 MiB compiz (73)

Anonymous said...

Dave,

I love reading your posts. One of the clients I support is a small city with about 12 users spread between the library, city, and police department in one building.

They are (as they should be) very conscientious of the budget and are always looking for ways to save costs.

Their big hold-up with using a linux-based solution revolves around two applications:

One is called Springbrook. (For all I know, you may use this in Largo) It's a do-everything app for cities that covers meter reading, business licenses, and a lot more I know nothing about.

The second app is a program used by the police department for the digital video in their patrol cars.

I know you've mentioned in the past that you have a few windows servers and have users setup to launch applications from their Linux desktop from the Windows servers. I'm wondering what you use in Largo. Is your city management package a windows-based program, or do you have a Linux solution? Do you just manage city machines, or law enforcement too?

Keep up the great work!

Dave Richards said...

darkpixel: Right now in 2008, it's virtually impossible to totally eliminate MS Windows in Government. So you focus on what you can control and still save a ton of money. Authentication, desktop (GNOME in our case), OpenOffice, Evolution and Firefox can be implemented. These commodity type applications are expensive, and the Linux versions run well. A move from PCs to thin clients saves 50% right off the bat as well, so there is massive savings there as well. As an example, compare a $650 thin client with a 10 year duty cycle ($65 per year) vs a $1200 dollar PC with a 3 year duty cycle ($400 per year). That doesn't even include operating costs. When I spoke about never being able to get rid of MS Windows, this is because of other agencies and at the state level. People send horribly proprietary documents and files from time to time, and you always have to have a way to open them. 12 employees would be a piece of cake to deploy. Thin clients to everyone, then have one Linux server which logs them in, and gives them Firefox, GNOME, media players, Evolution and OpenOffice. Then build a second server with Win2003+ and install those departmental and specialized applications and use Rdesktop to connect to it. A icon from the GNOME desktop goes right over to Windows server, and the software opens as it always did. Performance is excellent; in our case we even have an engineering CAD package running in this manner. Even 12 concurrent users on Linux is barely a blip, we have gotten 300 into mid-sized HP servers.

Anonymous said...

Thanks for the info Dave. The tough part will be convincing my boss. The excuse for paying the 'windows' price tag is always that he needs to be able to hold someone's feet to the fire when something goes wrong. (I reassure him you can get support contracts from RedHat, Canonical, etc...but he hesitates.)

Thanks again for the info.

Unknown said...

Dave, just out of curiosity - on what kind of network is this running? I'd guess some sort of Ethernet? Is everyone on that network in the same building or does the city have its own fiber network?

You also mentioned some low bandwidth sites - are they connected via ordinary cable/DSL ?

Thanks for a very interesting blog.

Dave Richards said...

Johan: We have 1Gb ethernet fiber optic lines running to all of the buildings close to City Hall. Then we have Cat 5e to the desktop and the thin clients are running at 100Mb. Our more distant sites are on the best technology possible based on the number of users and availability of technologies. That includes DSL, Cable and Frame. Users on the fiber are all running native X, everyone else is running NX. Even with X and all of the keystroke chatter, it's only 5% utilized and works well.

Anonymous said...

Online Buy Viagra for women Buy Viagra online
[url=http://wiki.sevenhats.org/display/~buy_viagra_online_uk]Online Buy Viagra canada Buy Viagra pfizer soft[/url]
Online Buy Viagra mexico Buy Viagra mexico http://udig.refractions.net/confluence/display/~boys_viagra
vertiloapolsas

Anonymous said...

[url=http://vacationhomerun.com/Community/members/zoloft-blob-87/default.aspx]zoloft blob[/url] [url=http://birinciyim.com/members/indian-tadalafil-29/default.aspx]indian tadalafil[/url]
buy ultram tramadol mg tablets Vetrinary AND dog AND tramadol
http://comunidade.salvemassereias.org/members/flomax-plus-70mg-64/default.aspx any drug effects side there viagra
[url=http://ivoroling.nl/members/cialis-princiapl-investigator-65.aspx]cialis princiapl investigator[/url] [url=http://fbcrecovery.com/members/mexican-pharmaceuticals-84/default.aspx]mexican pharmaceuticals[/url]
cvs pharmacy sale flyer archives can cold sores spread by hands
http://previewcommunity.wnetwork.com/members/viagra-soft-32/default.aspx affiliate casino internet
syphillis zithromax google4534553453322