Software Usage Tracking

Time has allowed me to put in some code to the new support portal for tracking software usage. We'll easily recoup this time in three areas. 1) the amount of time support spends looking for this information in the various license files 2) support is now one click away from being able to log into the server running the software without having to look through multiple sheets of information 3) we will be able to now begin the process of reducing software packages that are no longer being used.

In the shot below, the software packages are searched from /usr/share/applications and matches appear on the primary UI. When clicked the detail screen opens and displays a bar chart of the current month which indicates the total number of times users clicked on the icon. Below that it displays all of the users that are licensed to use this software. You can click on their name to pull up the user detail screen. At the bottom we have technical information concerning the location and name of the .desktop, the launch script and the remote server which runs this particular package. We also have stats on the total number of clicks for the day, month and year.



When you click on the Server IP button, it detects if the remote server is Windows or Linux. If it's Windows it opens a Rdesktop connection. If Linux, it opens a command line window.

If the system admins click on the launch script or .desktop button, it opens that file in gedit for review.



There are just a few more areas in my head to write, and then I'll spend time cleaning up the code and making the screens look nicer. The feedback from my coworkers has been positive, and it's great to hear that these few hours are freeing up their time by making things easier.

Happy weekend.

The iPad Villagers At The Gate

The picture above is how I feel lately concerning the user community and using iPads. If you support end users, I'm sure you know what I mean. :) It should have been fairly painless to implement as a thin client, but unfortunately the NX 4 release is taking far longer than anticipated. We have not yet seen anything ready for deployment on tablets and because of the torches we are going to begin testing under another design. This should give the users the same presentation but it's a bit clunky, we are hopeful this a short term (very!) design.

Instead of the tablets connecting to the GNOME/Linux server directly with NX 4, we are going to add a VM instance of Microsoft Windows and use the RDP protocol. Users will use a RDP client, connect to Windows and once authenticated will immediately start the NX client on Windows and connect to GNOME and give them the new desktop. The diagram below shows how it will work. This is -very- nasty, but unfortunately we have to move *something* into general testing.



Another quick project came to me, the desire to make Beagle scan small subsets of our data and make it easier for end users to find just documents under a certain category (folder). So I hacked our Beagle front-end and added "Spotlight Searches" which simply reduce the number of documents returned in the query. Users cannot build complex queries on the command line, things need to be very simple and this meets that goal. If they want to scan just our City Policies for certain key words, it's just a few clicks away.

Desktop Home Stretch

After the technical issues of the last few months, the GNOME desktop upgrade is finally on the home stretch. We're very excited about this deployment and the users seem happy with the results. Now that the kernel issues are resolved, the server is VERY fast. Even with 50 users, avant-window-navigator is fully rendered in about 3-4 seconds and ready for launching software. Several months ago I spent time firing all of the processes in certain orders and multi-tasking them to get the fastest possible response. I'm pleased with the results.

We're still waiting on the next NX release so that I can test this over lower bandwidth devices. Sadly that is taking longer than expected, but one can always be hopeful of a release date soon. I'm not expecting any problems with thin clients, the area that will need testing is tablets.

One of the biggest flaws that we have on the older GNOME desktop is clear and provable records of who is using what software. We have so many little MS Windows applications and it's difficult to even know how often they are *really* used. Our Windows apps and Linux apps each go through a common set of launch scripts so it was very simple for me to dump each desktop click to a CSV file (below). We now have an instant audit of all activities. This data can be reviewed in a spreadsheet, and also will very soon be available from our support portal UI.

[ Very simple data log of each users clicking activity]


As time allowed, I have started to hook up all of our flat files to the support portal and move the [ Software ] tab past being vaporware. When the portal is started, it reads all of the files in /usr/share/applications and finds all of those active on the desktop. It then reads in our license files and builds an array with this data. Now our support staff can search for programs, hover their mouse over the icon and see exactly what operating system and IP is being used. It also indicates the users that have licenses to run the various packages. (as seen below). This is a huge time saver for them. Everything is dynamic, and eliminates having to use another tracking software package for this purpose. Start up the portal, and it's reading live data immediately.



I built a quick UI to get my head around the information that I want to display when you click on the software detail button. Please, please...no UI critique. :) The goal in this first pass is to just work out the functionality and then I'll do cleanups later. Using the flat files described above, the UI will render a chart of the number of clicks of the software package in the current month (purple). You'll then be able to use the arrow buttons to navigate through the months and look for usage. It will display buttons for all users that have access in the middle section. The lower area will give more technical information concerning the app, including the .desktop and launch scripts used. It will allow you to click on buttons and edit those files with gedit. When you click on the IP address of the server which is used for the various packages, it will either telnet or rdesktop to that server for review and to make it easier to kill stuck processes. Right now, support has many sheets of paper with this information for 250 software applications running on around 25 servers. This will be a huge time saver. The lower section also will display additional usage information to assist in determining how heavily software is being used.



The last few days have been spent going through all of the icons on the production desktop, figuring out exactly what task they perform and then contacting users to see if they still use the software. I'm then upgrading the icons, testing them with Seamless RDP and updating the artwork. I hope to be finished with this process by Monday.

Everyone is sensing this project is nearing completion, and I think we'll have a deployment to be proud of.

And There Is Java Sound

Sometimes one creates a post just as a way to document something that they do not do that often. In a few months when I have to do this again, it'll be here. That is the case today. :)

We are getting our first application that uses Java to deliver sound, and of course it has no way by default of delivering to the thin clients. After some Googling and poking around, the process is not terrible. I installed Java 1.6.27 into /usr/java and then got the open source version of Java in RPM format and extracted the two PULSEAUDIO files:

rpm2cpio java-1_6_0-openjdk-1.6.0.0_b17-7.3.x86_64.rpm | cpio -ivd ./usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/jre/lib/amd64/libpulse-java.so

rpm2cpio java-1_6_0-openjdk-1.6.0.0_b17-7.3.x86_64.rpm | cpio -ivd ./usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/jre/lib/ext/pulse-java.jar

I then extracted the sound.properties file as an example of how to add the lines for PULSE.

rpm2cpio java-1_6_0-openjdk-1.6.0.0_b17-7.3.x86_64.rpm | cpio -ivd ./usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/jre/lib/sound.properties

You then hand move these files into the appropriate folders in the real Sun release, and merge in the PULSE lines into the sound.properties file. Restart your browser and Java sees PULSE and considers it a sound card. Very nice, we now have sound.

In the shot below, the vendor software is displaying the sound card and the clips play nicely.

Happier OpenSuse Days

This week we scheduled, installed and rebooted OpenSuse 11.4 with the 3.0.4 kernel. We broke the infamous rule of technology and also changed from the kernel-desktop to desktop-default; which means now that it's working beautifully we are not 100% sure which of the two issues resolved the problem. If one had more hours, we'd prove or disprove both theories but at this point we are going to enjoy the success and just move ahead. We've used both kernels on other servers and both have worked fine. Previously on this new server disk performance was terrible after user loads increased and now it's absolutely blazing in speed. Even with 40+ users, avant-window-navigator is starting to open in about 3 seconds even over remote display. You can barely even notice that 40 concurrent people are working -- in other words it's working as I would expect from Linux. I have solicited our users for more beta testers and hope to get us in the 50-60 mark by next week.

Other updates:

No NX updates this week to test, so I'm going to re-install and tinker with the current beta to see how it works on our new Wifi network. Still have those performance ideas in my head, waiting for the new Linux client updates in order to test them.

A few Evolution patches to test were received from Novell. They were compiled for SLED 11 SP2 which is not yet released. So I built a quick server with old hardware and tested them. The patches were not entirely satisfying, but at least now we know their status. Emailed results back to the developers.

We have a requirement for our first Java applet to work with PULSE, which I have never done before. A quick Google indicates it's possible, possibly via the Alsa layer. Time to tinker and better understand how this all works. Right now the sound from Java is very nicely trying to play on the server in the computer room. :)

I spent a little time hacking on the new Support Portal software and adding features that are useful to me (and others). It's now much easier to search for users and display their departments and to search by departments. With many hundreds of employees, it's almost impossible to keep them all straight. Next up, I'm going to write some quick code to keep track of what's in /usr/share/applications/*.desktop and log which users are clicking on which applications and how often. All of our icons pass through two common shell scripts (one for Windows, one for Linux) and it's going to be very easy to just write out data when they are clicked. This will allow us to better understand applications that can be removed from the network because no one is using them anymore.

Started to look at upgrading MoinMoin from 1.8 to the 2.0 (beta) release. Lots of libraries that need to be upgraded, and will require an operating system upgrade as well. Downloaded OS 12.1 (beta as well) for review and testing. Technology churn really keeps us employed, right?

Happy Weekend.

OpenSuse + Support Portal Code Deployed

The OpenSuse guys have been helpful in testing ideas to find the server bottleneck. I really don't stress over these things. Our older GNOME server is running fine and only those people that volunteered to beta test are seeing the performance issues mentioned in prior blogs. What's interesting is that even with the occasional freezes and slowdowns, they still prefer the new technology. All of the UIs are much nicer and easier and they really like avant-window-navigator. I'm sure we'll find it.

One of the things that is nice about hacking code is that you can picture ideas in your head and just write it and make it work. I love to see charts, stats and data. I also love having the computer monitor itself and report to us issues. Our Support staff has the same viewpoint. So I wrote the "Load" detail screen. The main portal displays in green the current user load for all of the big servers, and now when you click on it it gives you a chart with a much longer timeline. It then creates a front end to "top" and shows the top processes. The user name is a button, and you can click on it and the user detail screen which was already coded opens. It then hunts down all of their sessions along with information about their department. One can hone in on the sessions even further if desired and obtain information about the device they are using.

If any of the servers are over 10% busy for more than 10 minutes, a warning notify-send popup is generated and alerts all of us to review the issue.

The processes are GTK toggle buttons (marked in purple) and I'm going to allow our support staff to select processes and click the STOP button to shut them down. All of these features were available from the command line obviously, but this front end simplifies the whole thing greatly.



Up next for me: New NX 4 preview code (hopefully next week), testing the new WiFi with iPads, trying to get our Moin Wiki upgraded, trying to get 2 Evolution patches merged for SLED 11, installing and testing the 3.0 kernel on OpenSuse. But first, a three day weekend to recharge.

OpenSuse 11.4 Reload & Problem Is Back

Tuesday we moved our OpenSuse 11.4 users to a VM copy of the new GNOME desktop and we wiped the physical machine completely. We pulled the drives from the RAID array and put them in a new order, and then re-added a new RAID (1+0) partition. Our customizations are always placed in /u and /u2 so a few well placed tarballs and tweaks and we had a brand new machine with the same functionality as the old one.

Very sadly, the same issues have come back. I appreciated all of the ideas and tips presented in the comments area in prior blogs. But I do have a bit more information and it's very odd. This server should run 200 users easily, but as we get over about 20-30 and especially at 40 we see performance get very poor. Here is the new information: If I vi /etc/services it blinks for a few seconds and then opens. If I copy /etc/services to /tmp/services and /home/services and vi those files...it opens immediately. So there is some kind of contention or lock on the /etc/ directory. This contention seems to be the core of the problem. So many services are constantly looking at those files, and they are somehow bottlenecked. If you have any ideas to assist, the bug report is here.


Networking is still sub-optimal; note the dropped RX packets.

eth0 Link encap:Ethernet HWaddr 00:1C:C4:93:DF:72
inet addr:128.222.99.243 Bcast:128.222.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12191295 errors:0 dropped:73239 overruns:0 frame:0
TX packets:12731836 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5112551787 (4875.7 Mb) TX bytes:10726315924 (10229.4 Mb)
Interrupt:16 Memory:f8000000-f8012800

eth1 Link encap:Ethernet HWaddr 00:1C:C4:93:DF:74
inet addr:172.23.1.235 Bcast:172.23.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:73278 errors:0 dropped:238 overruns:0 frame:0
TX packets:8009 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15639151 (14.9 Mb) TX bytes:827822 (808.4 Kb)
Interrupt:17 Memory:fa000000-fa012800

There are a few more steps that we can take, including installing the 3.0 kernel to see if that helps. We might have to start looking at other distributions if we don't see progress soon.

Other projects continue: Writing some code for the support portal, testing NX, looking at upgrading our Moin Wiki, WiFi upgrades. I also have been poking at some ideas to improve NX performance to our thin clients, will blog about that if it pans out.