Whew!

I just looked and my last blog was March 27th; The last 1+ month has been very busy.  Not a lot of things took place that warranted screenshots or R&D updates.  OpenLDAP and Zimbra deployments both occurred and took most of my time.  Status updates:

OpenLDAP
The change to centralized passwords went off very well.  During my prep work, I created a "Password Sandbox" application that allows you to test various passwords before actually making the change.  This proved to be very popular and effective.  Part of my time was then allocated monitoring all 800 of our users to ensure they all changed their passwords and sending out daily status reports to work on 100% completion.  It's now all done and our passwords have been hardened to current metrics.

Support Portal Application
I have been putting in some patches and features for our support staff.  They could see at a glance users with passwords that still had not been changed, and made it easier to create and upload LDIF records.  We've reached a milestone with this functionality where it's very simple to create new employees in LDAP.  We also can now auto-provision new people in the Zimbra post office with just a single click.  Within 10 minutes of checking the button, the user has email.

LDAP Sync To Zimbra.
Zimbra auto provisions email accounts, and I wrote a small python script that runs at 1am to push any updates that occur from OpenLDAP to Zimbra.  Phone numbers and job titles change, and all of this information is now synced.  Support staff only has to change the information in one place, saving us time.

Zimbra Is Live
After a long ramp up time of testing, we put Zimbra live and it replaced Evolution/Groupwise.  It was kind of sad to move away from Evolution, but unfortunately we just could not get bug fixes from Novell/Suse under our support contract in a timely manner.  Zimbra was deployed with virtually no problems.  We built one server and have about 250 concurrent connections via the web interface and then an equal number of connections from mail-notification(IMAP) on the lower panel to alert them of new messages for a total of 500+ connections.  Zimbra is flying and you can barely notice the difference with a full load.  Fast and crisp.

In the past our server which provided Firefox would have about 100 concurrent users, now that users are accessing email from a browser this has jumped to about 250.  The server took this load without a hiccup and is working very well.  With 100 concurrent the machine would be about 10% busy and now it's running about 15%.  We scaled this machine with the expectation of greatly increased Firefox loads and it's working like a champ.

And The One GOTCHA
There is always one, right? :)  LibreOffice (and OpenOffice) have features embedded in various places to hand off the current document, or the current document converted to another format directly to your email software.  This is a great feature because it eliminates the need for users to convert to other formats prior to sending to outside.  A document can be created in ODT format and retained here in that manner, and then converted to DOC on the fly...and the DOC is never saved to the local file system.  In case you didn't know it did this, here is where the magic happens:

Now the bad part, it basically does a mailto: to your email client and uses &attach which it turns out is not officially in the RFC and is only available in certain email software...including Evolution.  But many/most web based email software does not allow or support it.  I can certainly see that this could be a bad exploit.  Someone could create a mailto tag that requests files of an unsuspecting users desktop. 

You cannot just tell users to drag and drop the file out of Nautilus because in many cases, it's generated as a temp file and is never saved for them to be able to find.  You can't expect them to open Nautilus in /tmp and *maybe* find their file out of the hundreds of other temp files from the day.  Clunky to say the least.

Zimbra supports HTML 5 drag and drop reception, so I worked up a prototype idea.  Jasper and Federico were so kind to help me with the drag-and-drop mechanics of GTK and we're testing this concept now.  When a temp file is generated and needs to be attached to email from LibreOffice (and other applications if it pans out)...a popup window appears on the left edge with a thumbnail of the document.  The end user can then grab the document and drag it into their composer window.  In the shot below, user has selected the option to "Email as PDF" and the popup appears with thumbnail and Zimbra composer window opens (PURPLE).  The user then drags the file into the composer (RED). 



In the coming days, I'll be testing this concept and soliciting user feedback.  If it seems to be working, I'll connect it to the rest of our applications that previously auto-attached files to Evolution. 

Next up for me is creating LDAP groups and then getting Alfresco to auto-provision users and put people into those groups.  We're at the breaking point around here of maintaining documents only on a file system and are in need of a management system.  We'll be pilot testing this in IT to see how it works.

Other issues: 64bit HP thin client is sitting here for me to test, various portal changes have been requested, continued Zimbra troubleshooting and support, various Java/Flash/Firefox upgrades. 

LibreOffice 4.0 Live, Other Projects

Monday night we had a window of opportunity and installed LibreOffice 4.0.1 for all of our users.  The install went cleanly and Tuesday morning we received NO calls with anyone having problems with their documents.  Considering how many users we have and the number of file formats they are opening, that is wonderful news.  Congrats to the developers and QA people!  First launch of LO 4 offered them a custom dialog with new features, and the opportunity to receive a copy of LO for Mac/Windows for home use.    A high number of people requested media, and we're happy that they are are using it on their own.

In the coming days, I'll be adding desktop triggers for Corel Draw, MS Visio and MS Publisher files so that they just open automatically with LO 4.0.  This will at least allow people to view documents that we receive from outside contacts that assume everyone has licenses for such software.  Very often, the files just need to be *viewed* and not edited and this eliminates having to spend money.

I submitted a few feature requests to the LO developers for enterprise users.

We have been working on moving all of our user passwords into LDAP and that is going well.   I have been working on some issues with syncing data between OpenLDAP and Zimbra, and making progress.  I put some code into our support portal software to allow our staff to create new LDAP entries from a LDIF file without having to do so on the command line.  A few clicks now and it's all done and pre-checked for validity and has all of the required attributes for our various downstream needs.

I have been looking at auto provisioning user accounts in Alfresco as part of a pilot program.  Alfresco needs user groups in LDAP which were not required with Zimbra.  So I have started to facilitate this change and make it easy to put users into departmental and divisional groups.  Once done, I'll try and get Alfresco to automatically create all of our users.  We are going to explore using Alfresco in IT, and once we have a work flow...show it to the other departments and try and obtain funding for a support contract and possibly adding an employee to be the custodian of the data.

The HP 5745 thin client was discontinued and we ordered the replacement model and it has arrived.  I'll spend some time checking it out and will take pictures and review it here on this blog in the near future.  Never enough hours in the day sadly.

Very happy to see that SuseCon 2013 will be in Orlando again this year.  It's a no brainer that we'll be attending and it will be nice to chat with all of you in person.

LDAP Finished, Zimbra Just About Ready And More

I haven't blogged in a good while, but all projects are advancing nicely.

Open LDAP
All records and data have been loaded and are working.  After reviewing Alfresco, Zimbra, WiFi and proxy device, I believe we worked out all of the required information and it's all ready to go.  We always knew this would reduce work, but normally would need a specialized employee to run the infrastructure.  Since we aren't getting more staff, we have to get creative in solving these types of problems; how best to deploy technology that really should have additional IT staff?

LDAP Portal Integration

The solution as I touched upon last blog was to write some code so it's maintained from our Support portal software.  Our Support group enters the information and it's verified and formatted consistently and then required fields are uploaded to OpenLDAP.  In order to have multiple levels of fallback, all information is stored in LDIF files, staged and then uploaded.  If something happens to a user record, we can always recover the data.

The shot below shows the portal screen information is entered and saved in LDIF files, it's then compared against LDAP and allows for update when data has changed.  This code is live and seems to be working. 

With the contact information working, the next step was to make it very simple to grant permissions to various authentication functions.  So under the Licenses tab a small section was added with checkboxes.  Click them and update LDAP and everything else is handled automatically.  Available options are Zimbra, Firefox (Proxy), WiFi and Alfresco.  We are enabling auto-provisioning on the various sub-systems on our network.  No more having to maintain hundreds of user accounts, efficiencies gained.

City Directory

As mentioned in my last blog, the first use of the LDAP/LDIF data was a quick "City Directory" screen that allows users to search by name and job title.  This information was already available in a less user friendly way via Evolution and in Zimbra, but they just aren't locating the data.  The support portal tracks software usage, and this application has been used 678 times already in about 2 months.  This tells me that people were really struggling to find this information and were probably maintaining their own phone number documents.  Very happy to see this much usage for a few hours of coding.

LibreOffice 4.0:  I did not install 4.0.0, and wanted to let others find any major bugs that might exist.  4.0.1 is just a few days away, and I'll install when it's released.  It will be nice to have initial CMIS, and additional document filters.  I don't expect any significant issues.

NX and Mac Clients: We have been struggling with the fact that production NX doesn't have Mac clients for Lion/Mountain Lion.  More and more users are getting this operating system and no user friendly solution is available in the short term.  This issue will be resolved with NX 4, which is still a beta product.  What we are going to do is create a virtual copy of our GNOME server and install the 4.0 beta server piece and use that for Mac users until the product is officially released.  Unfortunate loss of man hours for me having to do this, but Mac usage seems to be increasing all the time for home users and we need to get them working.

Zimbra Email:  800 accounts were created automatically with Auto Provision and LDAP, which is simply wonderful.  Within a few minutes of the connection forming, they were all done.  This would have taken days to do by hand.  We're testing the infrastructure right now with web browsers, phones, tablets and pushing email and everything seems to be working.  We're targeting a mid-April deployment.


Quick Updates: Patches and QA for Java, Firefox, Flash and Adobe Reader.  At least one of these 4 products seem to have a security update every week.   HP discontinued the t5745 thin client and released a new 64bit model, will have to find some time to check it out and look at how to get our code installed.  Never enough hours in the day sadly.

Keeping Busy With Projects

I just looked and it's been over a month since my last post.  Christmas and some vacation consumed some of that time, along with various projects.

Upgrades, Patching

A lot of software had security bulletins so I have been QAing upgrades and getting them installed.  Firefox, Java, Flash and Acrobat Reader all got updates and they have been pushed into production and are working fine.  Upgrade churn makes me really appreciate being able to upgrade from one centralized server.  Pushing updates to workstations this often would consume many more hours.

OpenLDAP

I appreciated the comments and ideas from my last post, but we are going to continue with using OpenLDAP.  All of the software we use has been tested with and support using this directory server.  I kind of had to make the rounds on all of the places we are going to use LDAP and make sure that I got all of the fields that will be required.  For instance, some software packages require that the first_name and last_name be split into unique fields, so I had to add them to the database.  I believe now it's nearly finished and things are working as expected.  Now to begin reaping the benefits of this data as noted in our other projects.

City Directory

Both our nearly retired Evolution email software and the upcoming Zimbra software have the ability to search the address books for City employees.  And both software packages allow for advanced searches.  However, it's been my experience that unless it's one or two clicks, there won't be success with end users.  They don't want to have to write commands to search for users, and won't even with clear instructions.  Sometimes they want to search for job titles, and sometimes they want to quickly get a listing of all users in one particular department. Because users were not having success doing this with software, there were pockets of users creating their own paper copies of our City Directory/Address Book and manually doing updates.  Obviously this isn't efficient.  So to show off the LDAP data, I wrote a quick City Directory application to allow for quick and easy searches of the data, and to display employee photos.  It's finished and we're seeing that it's already being used by a lot of people and usage is growing.  Clearly this was an improvement area.  The shot below shows the appearance:  From the main screen you can enter a users first name, last name or job title and a search will be performed.  You can also display all users in one particular department.  Clicking on the user button brings up the detail screen (BLACK).  The user screen shows all of their information, and also allows you to enter your own note (PURPLE) for this particular person.  You can also check to see if they are online (RED) and if you so wish, send them a request to remote control their screen (BLUE).  Very often we have employees who are supporting their own software systems or doing training, and it's been invaluable to offer the ability to remote control sessions from any City workstation in our multiple buildings.

In thinking how this would work, I decided against this doing a query to LDAP directly.  Instead it loads the source LDIF files once at startup and loads all of the users into one big array.  That way people aren't hammering LDAP with queries and response is immediate.  It's not a big deal if their data is a few hours old.  Possibly I'll add a few lines of code to rescan the data from time to time and build a new array, but I don't think it will be a problem.

** Yup, the screens need some UI love, as time allow. :)

Support Portal Creates LDIF Files

As mentioned in my last blog, we aren't going to get a "LDAP Administrator" employee, so we have to make due with our current staff.  The best way to handle this in my view is to have multiple avenues of backup so that if something is errant, it can easily be repaired.  There will be lots of us doing entries, and the possibility exists that mistakes will be made.  The design is that our support portal creates LDIF files, which are stored on the server.  There is a button (BLUE) which compares this data to the LDAP information and alerts them that fields need to be updated.  At that point, a button will be clicked and the information will be logged and then inserted into LDAP (PURPLE).  Having open source tools like python and Glade make these customized screens a snap.  We can mold them to our current staffing size and skills, which is the case here.

PDFMod And MIME Helpers

PDFMod is a great software, but has gotten a bit crusty because it's not being maintained.  It's not able to read in newer PDF files, of which we are getting more and more from outside contacts.  We started to get an increase of support calls about PDF files that failed to import. I made a small change to the MIME help applications that open when you double-click on files, and we now offer a checkbox which allows the users to "downgrade PDF" before opening in PDFMod.  This seems to solve the problem and our users were happy with the work around.  PDFMod sure would be a top notch Linux application with just a small amount of love.

Zimbra

After a long process, we were able to purchase Zimbra and I literally got the final license key as I was typing this blog.  The production server was built with Ubuntu Linux and was patched, Zimbra was installed and I'll put this key on today to eliminate the nag messages.  I'm working with the Zimbra people on the best way to authentication and provision from LDAP and once that's done we'll start a wider beta testing process.  It's exciting to have new email and calendaring and I think the users will be happy with the ability to gain access with any browser. 

Other Projects

Testing LibreOffice 4.0; found and submitted several bug reports..many of which are already fixed.  Starting to look over Alfresco again...this project will kick off again in a few months.  Trying to get my head around GNOME 3 with thin clients, it doesn't support remote X as does GNOME 2...are we going to be moved in the direction of MATE?

Infrastructure Readied For LDAP

Sometimes there are circumstances where we know that certain technologies will make us more effective, but we don't have people on staff to implement.  This was the case with LDAP publishing of user accounts.  Clearly it's wonderful, but no one is on staff to properly maintain it.  It's easy to fear the situation where even if one person gets it working, if they are out of the office you have the potential for longer down times if something breaks or fails.  Do you really want System Administrators punching in phone numbers and making updates?  We reached the point where it's needed, we have too many external applications that need shared information.  How best to deploy?

After consideration and discussion we decided to create a screen in our support portal that allows all IT staff to make modifications to basic information and it creates a LDIF file which is housed on the GNOME server.  While we perfect our work flow, these files will be uploaded into the LDAP server by our admins, with the goal of automating that once it's been proven.  This approach is clean to me because we have all of the "source" files out of LDAP.  If something gets damaged, it's easily repaired and recovered.  So I wrote a little vcf2ldif utility that sucked out all of the data out of Evolution/Groupwise and created LDIF files. 


The support portal software already had a "user detail" screen, so the fields that we needed were added to another tab and are easily editable.  When the save button is pressed, it generates a LDIF file with all of the exact formatting that we want and fields locked down from free form text entry when appropriate.  Very easy to use, and integrated into the software used mostly heavily by out support staff.  I saw that LDAP supports a JPG image, so our security guy downloaded all of our photos from our badges and I added a place in the UI to add them.  We can now see the pictures of all employees, which will assist us with identity confirmation.

Pictures were also thumbnailed automatically when doing a user name or department search:

After trial and error and a few runs of the vcf2ldif utility, the files were compliant in all fields.  So I did a cat *.ldif > merged.ldif and then ldapadd'd them all into the software with success.  It's a very exciting step, all done easily with open source tools.


And the data was verified with a LDAP browser, and appears as expected:

My current area of focus is now accessing this data from our first new project: Zimbra.  Zimbra supports auto-provisioning, whereby it verifies if you have a mail: LDAP entry and if so, it verifies your password.  If this is the first time you have logged in, all of the phone numbers are downloaded and the account is automatically created.  This will save our support staff a lot of time.

LDAP will also be used for our Wifi, Alfresco and our web proxy appliance in the coming months. 

There are always many ways to accomplish goals, but I think we have found a good balance with our staff size, budget and existing skills.  I'm looking forward to seeing this all deployed.

Changes To Portal, And Data Mining

It's an aspect of my personality -- I love data and finding new ways to analyze it.  In the case of our GNOME implementation I want to see exactly what software packages are being used and by whom.  There are two benefits to this:  1) being able to focus on the software packages used the most  2) Being able to reach out to users when data indicates they might be having problems.     Very often in a corporate/enterprise environment users will not call for support and their frustrations will grow, so our support portal has given us the ability to proactively make contact.

In the last few set of updates that I pushed, the "Usage" tab of the portal has been implemented and is now providing interesting data.  The UI and buttons still need some alignments and work, but the data is there for viewing.  On each refresh cycle it's now counting all clicks for the day and sorting them into the top four categories.  And then underneath these categories it's how showing the top six users and their number of software clicks/launches.  In the case of LibreOffice this might just mean they are editing a lot of files; it also might mean they are having a problem.  In the shot below, I can now see the number of launches of "Evolution" for instance (marked in green) and then the blue shows the users underneath.  There were some users with lots of launches and one of them is having problems with quota.  The other user seems to be working in a manner of only opening one software application at a time and closing it before opening another.  We are reaching out to both employees and offering to help.   With these most recent changes, I've reached a milestone now where all features that we wanted to add are now complete and I'm moving to other projects.

Projects:  Tomorrow we are working with a vendor to design and implement a LDAP server, and then will be connecting it to our Zimbra install and testing this type of authentication.  Once that's in place, I'll connect it to Alfresco for similar testing.  I also will be working again on the Ubuntu thin client updates for our workstations.

Support Portal, Licenses, LibreOffice & Zimbra

Things have been busy since my last blog.  Along with normal day to day support items, I have been advancing several projects.  I have had some time to multi-task in making changes to our support portal.  We have found that the time that I invest in improving that software is easily recouped down the road.  For many years licenses were granted via files and the command line.  Things have gotten more complex since this infrastructure was first deployed.  We have more staff members, we have lots more software packages, and licenses are being moved around more than ever.  So I created a new way to hold the licenses and wrote a little script to make the conversion and integrated it into the support portal.  Now our support staff can go into the software package (as loaded from the .desktop file) and with a few clicks add permissions for users to launch this software.  They also can point the license to the physical paperwork; and a thumbnail is generated to show the document.  (as seen below).

I also mined into the log files where we track usage and added some fields to display stats of our various packages.  We have so many software packages now, and very often people buy things and never use it...so now we'll have more data with which to review if software should be removed from servers.

Everything seems to be working well in our sandbox test area.  With just a few clicks licenses are granted and revoked, all changes are logged and the license goes live immediately.  The user can click on the icon and it works as soon as the save is made.

As part of this code, I added a "Usage" tab from the front UI of the portal whereby we can watch in realtime as icons are clicked.  We also can see which applications are being used the most and by whom.  I'm going to add some buttons below the software categories that will show which users launched these applications the most during the current search window.  When viewing the current day, this might indicate that they are having some problems or technique issues that need our help.
  (Very often people never call...even when they are having serious problems.  We want to identify these issues and be proactive)

Other projects have continued:  I have been testing LibreOffice 4.0 Alpha along with some other employees, a few issues found and bugs submitted.  We have the hardware now for the final deployment of Zimbra, so I installed Ubuntu 12.04 server and applied patches and got everything running.  Next week we connect it to LDAP and we will test accounts and passwords. 

GNOME/Exceed, LibreOffice, Portal, Zimbra

Things have been very busy since the last blog.  Lots of concurrent projects happening and some long term design elements falling into place.

GNOME/Exceed

The software that allows iPad tablets to connect to GNOME was moved over to the production machine and is now in the hands of some beta testers.  The speed is great, and things are working really well.  From home, I can tap a few buttons and my work GNOME session appears on the tablet over my cable modem and is almost as fast as being in our City Hall building.   I sent off a few suggestions for making it easier to work on a multi-touch device with mouse based software and hope to see some of these ideas in future client releases.  I'm anticipating that we'll buy some licenses at the end of the trail.

Portal Tablet Support

Being that users will now be logging in with tablets, I wanted to change our in-house support portal to correctly log these authentications and make it easier for our staff to see where and how users are logged in and be able to send them remote control requests.  Very often users call and aren't really sure how they are logged in, and the software now helps us see exactly what's happening.

I always have more ideas than time, and have wanted to work in this area for a long time.  From the portal when you enter the user name, it hunts the network for their active logins.  With my changes made, it understand if they are logged in with a tablet and displays it in the right area and layers a shot of their screen over an iPad image.  Previously to perform functions on a session, you would click on the screen thumbnail and go to a detail screen.  I added small shortcut buttons below the screenshot to eliminate another few clicks.  [ I understand certain UI decisions and artwork could be better.  Time doesn't always allow for this level of design.]   It works, we use it all the time, and it's constantly evolving

I also spruced up the code to detect users that are logged in multiple workstations and this makes it easier to find the right session.  Users also can log into workstations and tablets at the same time too.   This all is clearly displayed.  In the shot below, I was logged into two workstations and both were located.

LibreOffice

 3.6.3 came out and this is the best one yet.  The earlier releases had some install bugs that I worked around.  This one installed cleanly and we have no reports of users crashing and import filters are working better than in earlier versions.  If you have been holding on 3.5, now might be the time to look over 3.6.

Zimbra For Email

After a lengthy process the Governance Committee has asked us to move to the Zimbra email software.  This software runs 100% in a browser, and this means that Evolution/SLED and Groupwise/SLES will be retired.  We've had Groupwise since 1994, so it's a major step for us.  Especially in the 1990s, Groupwise was away ahead of its time, but it fell more and more behind and we couldn't get Evolution patches and features fast enough to keep up with demands.  This upgrade will happen sometime in early 2013.

Other projects: Teaching myself OpenLDAP and learning how best to get it integrated, looking over Firefox 17, keeping plugins current and continued work on the new Ubuntu thin client operating system.