Thursday, March 20, 2008
BrainShare Wednesday
The Wednesday keynote was, indeed, a bunch of demos. It was also mostly pointless as far as the technology I'm concerned with. Lots of GroupWise (don't care), lots and lots of PlateSpin (can't afford it), lots of Zen (not the bits I'd use).
That said, the new GroupWise WebAccess is gorgeous. I wish Exchange had their non-ActiveX pages look that good.
TUT175: RBAC: Avoiding the horror, getting past the hype
Mostly about IDM as it turned out. Only minimally interesting from an abstract viewpoint about roles in general.
TUT 277: Advanced eDirectory Configuration, new features, and tuning for performance
I learned a few things I didn't know, such as the fact that each object as an "AncestorList" attribute listing who their parent objects are. This apparently greatly speeds up searching. SP3, coming out this Summer, will have faster LDAP binds for a couple of reasons. Right now Novell is recommending 2 million objects as a reasonable maximum size for a partition for performance reasons.
And also they reiterated something I've heard before...
TUT221: iPrint on Linux, what Novell Support wants you to know
A nice session from a mainline support guy about the ways people don't do iPrint on linux correctly. We're not going there until pcounter can run in linux, so this is still somewhat abstract. But, nice to know.
That said, the new GroupWise WebAccess is gorgeous. I wish Exchange had their non-ActiveX pages look that good.
TUT175: RBAC: Avoiding the horror, getting past the hype
Mostly about IDM as it turned out. Only minimally interesting from an abstract viewpoint about roles in general.
TUT 277: Advanced eDirectory Configuration, new features, and tuning for performance
I learned a few things I didn't know, such as the fact that each object as an "AncestorList" attribute listing who their parent objects are. This apparently greatly speeds up searching. SP3, coming out this Summer, will have faster LDAP binds for a couple of reasons. Right now Novell is recommending 2 million objects as a reasonable maximum size for a partition for performance reasons.
And also they reiterated something I've heard before...
You know how back in the NetWare 4 days, we said to design your tree by geography at the first level, and then get to departments? Um, sorry about that. It was great back then, but for LDAP or IDM it really, really slows things down.Yep. I took my first class for my CNA when 'Green River' was just coming out, or was just out. So I remember that.
TUT221: iPrint on Linux, what Novell Support wants you to know
A nice session from a mainline support guy about the ways people don't do iPrint on linux correctly. We're not going there until pcounter can run in linux, so this is still somewhat abstract. But, nice to know.
- The reason that some print jobs render differently than direct-print jobs, is because of how Windows is designed. Direct-print jobs render with the 'local print provider', and iPrint jobs render with the 'network print provider'. This is a Microsoft thing, not an iPrint thing. You can duplicate it by setting up a microsoft IPP printer (assuming you're not mandating SSL like we are) and printing to the same printer with the same driver.
- The Manager on Linux doesn't use a Broker, it uses a 'driver store'.
- The Manager on NetWare doesn't always bind to the same broker. I didn't know that.
- It is recommended to have only one Broker, or one driver store per tree.
- Novell recommends using DNS rather than IP for your printer-agents, check your manager load scripts.
Labels: brainshare, edir, linux, microsoft, netware, novell, OES, pcounter, printing
Tuesday, February 05, 2008
Exchange vs Groupwise
A post on CoolSolutions today quoted another blog about why GroupWise makes sense over Exchange. This is some of the same stuff I've seen over the years. A faaaaavorite theme is to point to mass mailer worms taking out Exchange, leaving everyone else fat and running.
On 1/7/07 I wrote about just this sort of thing. A quote:
The other thing I mentioned 13 months ago was 'migration events'. We're coming up on one, in the form of Exchange 2007. As the other blog mentioned, the hardware requirements for Exchange 2007 are a bit higher than for 2003. Speaking as an administrator with a sizable Exchange deployment, the requirement of 64-bit OS is something of a non issue since I'd be using one anyway. For a small office with only 200 users, though, forking out for Windows Server 2003 64 would be expensive.
Another point mentioned is that GroupWise can run on anything, and Exchange (especially Exch2007) won't. Again, as a mail admin for a largish Exchange system that doesn't matter to me since I'll be using newer servers to keep up with the load anyway. Again, for small offices who upgrade their servers whenever the old one completely bakes off, this is a bigger concern.
The other migration point is the Public Folders that Microsoft dropped in Exchange 2007. Or rather, made a lot harder to manage. Their users roasted their account managers hotly enough that Exchange 2007 SP1 reintroduces Public Folder management. We make some use of Public Folders, but I can see an office that makes extensive use of them looking at Exchange 2007 as not a simple plonk-in upgrade that Exchange 2003 was from Exch 2000. GroupWise doesn't have a similar concept to Public Folders (Resources might be, but only sort of), so this doesn't help GW much, but is the sort of event that makes an organization really think about what they're moving to.
As for productivity, we haven't had problems. Our Exchange has about 4300 accounts in it right now. This is supported by three administrators and a lot of automation. That said, during summer vacation season when I'm the only one of us three here I can go whole days without touching anything Exchange. It just works. This is a claim I frequently hear from GroupWise shops, so... Microsoft can do it too eh?
Another thing on CoolSolutions lately has been a few pieces on marketing GroupWise. In short, it makes more sense for Novell to pitch GroupWise as the #2 player than it is to pitch it as fundamentally better than Exchange. This has some good points. There are some markets that GroupWise is a better fit than Exchange, and the small, infrequently upgraded office is one of them. As are organizations looking really closely at Linux. GroupWise can very well be the #1 mail product in the Linux space, so long as Novell can convince people that paying for email services in Linux is a good idea.
I close out my previous post 13 months ago with a paragraph that still stands:
On 1/7/07 I wrote about just this sort of thing. A quote:
The days of viruses and other crud scaring people off of Exchange are long gone. Now the fight has to be taken up on, unfortunately, features and mind-share. In the absence of a scare like Melissa provided, migrations from Exchange to something else will be driven by migration events. Microsoft may be providing just that threshold in the future, as they've said that they will be integrating Exchange in with SharePoint to create the End All Be All of groupware applications. Companies that aren't comfortable with that, or haven't deployed SharePoint for whatever reason may see that as an excuse to jump the Microsoft ship for something else. Unfortunately, it'll be executives looking for an excuse rather than executives seeing much better features in, say, GroupWise.Which, 13 months later, is still mostly true. Mass mailer worms are no longer the scourge they used to be, and are well handled by commercial AV packages. Mass mailer worms even look different these days, preferring to infest and send mail independent of the mail client directly to the internet, thus neatly bypassing the poor meltable Exchange servers. The fear of mass mailers is FUD leftovers from years ago, not a current threat or reason to get off of the dominant platform.
The other thing I mentioned 13 months ago was 'migration events'. We're coming up on one, in the form of Exchange 2007. As the other blog mentioned, the hardware requirements for Exchange 2007 are a bit higher than for 2003. Speaking as an administrator with a sizable Exchange deployment, the requirement of 64-bit OS is something of a non issue since I'd be using one anyway. For a small office with only 200 users, though, forking out for Windows Server 2003 64 would be expensive.
Another point mentioned is that GroupWise can run on anything, and Exchange (especially Exch2007) won't. Again, as a mail admin for a largish Exchange system that doesn't matter to me since I'll be using newer servers to keep up with the load anyway. Again, for small offices who upgrade their servers whenever the old one completely bakes off, this is a bigger concern.
The other migration point is the Public Folders that Microsoft dropped in Exchange 2007. Or rather, made a lot harder to manage. Their users roasted their account managers hotly enough that Exchange 2007 SP1 reintroduces Public Folder management. We make some use of Public Folders, but I can see an office that makes extensive use of them looking at Exchange 2007 as not a simple plonk-in upgrade that Exchange 2003 was from Exch 2000. GroupWise doesn't have a similar concept to Public Folders (Resources might be, but only sort of), so this doesn't help GW much, but is the sort of event that makes an organization really think about what they're moving to.
As for productivity, we haven't had problems. Our Exchange has about 4300 accounts in it right now. This is supported by three administrators and a lot of automation. That said, during summer vacation season when I'm the only one of us three here I can go whole days without touching anything Exchange. It just works. This is a claim I frequently hear from GroupWise shops, so... Microsoft can do it too eh?
Another thing on CoolSolutions lately has been a few pieces on marketing GroupWise. In short, it makes more sense for Novell to pitch GroupWise as the #2 player than it is to pitch it as fundamentally better than Exchange. This has some good points. There are some markets that GroupWise is a better fit than Exchange, and the small, infrequently upgraded office is one of them. As are organizations looking really closely at Linux. GroupWise can very well be the #1 mail product in the Linux space, so long as Novell can convince people that paying for email services in Linux is a good idea.
I close out my previous post 13 months ago with a paragraph that still stands:
So, Exchange will be with us a long time. What'll start making the throne wobble is if non-Windows desktops start showing up in great numbers in the workplace. THEN we could see some non-MS groupware application threaten Exchange the way that Mac (and Linux) are threatening the desktop.
Tuesday, January 22, 2008
Distributed Identity (such as OpenID) and security
Distributed identity systems are hot these days. Open-ID has been around for a while, and Yahoo! just jumped on that bandwagon. Possibly to stick it to Microsoft, who is deploying LiveID. Blogger just started allowing non-Google logins for things like comments.
These systems work by splitting apart authentication (verify who you are) and authorization (what you're allowed to do). Single-Sign-On systems work this way as well, but these systems take that to a much greater scale. Once you've been authenticated by the trusted third party, you are authorized to access the specified resources. In the web domain this is easily handled through cookies.
I noticed this text on the LiveID page I linked to:
Lets take it a bit further. It would probably be easy to get LiveID working inside of SharePoint. Especially since a developer SDK has been released to do just that. This would permit LiveID's access into SharePoint. Handy for collaborating with colleges working for other companies or universities.
Now what if Microsoft managed to kerberize LiveID? That would make it possible to use LiveID to log in against any Kerberos enabled service, as well as almost anything ActiveDirectory enabled. It'd probably take a tree-level (or maybe domain-level) trust established to the foreign tree (LiveID in this case) to make it work, but it could be done. Use LiveID to log into Exchange with Outlook, or map a share. Use your corporate login to work on your Partner's ordering system.
This scares me. In principle, not just because it's Microsoft I'm talking about here. Yes, it can be a great productivity enhancer, but the devil lurks in the failure modes. Identity theft is big business now, and anything that extends the reach of a single ID makes the ID that much more valuable. Social Security Numbers to us Americans are big deals since we can't renumber those, thus we have to protect them as hard as we can. Until we get a better handle on identity theft, these sorts of "One ID to rule them all," systems just make me wince.
These systems work by splitting apart authentication (verify who you are) and authorization (what you're allowed to do). Single-Sign-On systems work this way as well, but these systems take that to a much greater scale. Once you've been authenticated by the trusted third party, you are authorized to access the specified resources. In the web domain this is easily handled through cookies.
I noticed this text on the LiveID page I linked to:
Microsoft's Windows XP has an option to link a Windows user account with a Windows Live ID (appearing with its former names), logging users into Windows Live ID whenever they log into Windows.I did not know that. Shows what I pay attention to. What this tells me is that it is possible to synchronize your local WinXP login with a LiveID. This causes me to glower, because I inherently trust my local system differently than I do miscellaneous web services. Yes, the authenticator is the piece I need to worry about as it is how I get to prove I'm me, and that's just in one spot. But still, one compromised account (my LiveID account) and everything is shot.
Lets take it a bit further. It would probably be easy to get LiveID working inside of SharePoint. Especially since a developer SDK has been released to do just that. This would permit LiveID's access into SharePoint. Handy for collaborating with colleges working for other companies or universities.
Now what if Microsoft managed to kerberize LiveID? That would make it possible to use LiveID to log in against any Kerberos enabled service, as well as almost anything ActiveDirectory enabled. It'd probably take a tree-level (or maybe domain-level) trust established to the foreign tree (LiveID in this case) to make it work, but it could be done. Use LiveID to log into Exchange with Outlook, or map a share. Use your corporate login to work on your Partner's ordering system.
This scares me. In principle, not just because it's Microsoft I'm talking about here. Yes, it can be a great productivity enhancer, but the devil lurks in the failure modes. Identity theft is big business now, and anything that extends the reach of a single ID makes the ID that much more valuable. Social Security Numbers to us Americans are big deals since we can't renumber those, thus we have to protect them as hard as we can. Until we get a better handle on identity theft, these sorts of "One ID to rule them all," systems just make me wince.
Wednesday, January 02, 2008
Where NetWare Fits
NetWare 6.5 still holds top honors in one server niche. Even though it is a 32-bit operating system. That niche is the "large file-server" segment. I define "large" as, "lots of data, way-lots of concurrent users". Yeah, that's highly scientific. But "way-lots" means "over 1000 concurrent" to my thinking.
We regularly run between 1200-6000 concurrent connections on our cluster nodes. This is a density that just doesn't happen all that often in the market. If you have 6000 users close enough together to all talk to the same file-server at LAN speeds using a protocol designed for file-serving (such as NCP, SMB/CIFS, or AFP), you're a big organization. 6000 is a large corporate campus, a large governmental entity of some kind, or a larger .EDU like us. Nationally, the number of 'large' file-servers like that is peanuts compared to the number of 'workgroup' (i.e. under 300 concurrent users) servers out there.
It is therefore no surprise to me that Novell is not devoting a lot of engineering to supporting the top end of this market. While it may pay well, there just isn't enough revenue coming from these customers to try and handle the hardest-to-test use-case: very high concurrency. I find it disappointing because I AM one of those customers (a larger .EDU), but I understand the business drivers supporting the decision.
For the moment, NetWare 6.5 (32bit) is the top-dog performance wise for our environment. That isn't going to stay true for much longer. It would not surprise me to find out that a Windows Enterprise Server (x86_64) with 16GB of RAM can out-perform a NetWare 6.5 (32bit) server with 4GB of RAM, simply due to the added room for a file-cache. What I don't know is how CPU-bound file-serving I/O is on a Windows Enterprise Server, that's the one area that could keep NetWare 6.5 (32bit) on top. I already know that OES2-Linux out-performs NetWare for NCP traffic, so long as you stay within CPU bounds.
For high-concurrency applications, as far as I know NetWare still wins.
We regularly run between 1200-6000 concurrent connections on our cluster nodes. This is a density that just doesn't happen all that often in the market. If you have 6000 users close enough together to all talk to the same file-server at LAN speeds using a protocol designed for file-serving (such as NCP, SMB/CIFS, or AFP), you're a big organization. 6000 is a large corporate campus, a large governmental entity of some kind, or a larger .EDU like us. Nationally, the number of 'large' file-servers like that is peanuts compared to the number of 'workgroup' (i.e. under 300 concurrent users) servers out there.
It is therefore no surprise to me that Novell is not devoting a lot of engineering to supporting the top end of this market. While it may pay well, there just isn't enough revenue coming from these customers to try and handle the hardest-to-test use-case: very high concurrency. I find it disappointing because I AM one of those customers (a larger .EDU), but I understand the business drivers supporting the decision.
For the moment, NetWare 6.5 (32bit) is the top-dog performance wise for our environment. That isn't going to stay true for much longer. It would not surprise me to find out that a Windows Enterprise Server (x86_64) with 16GB of RAM can out-perform a NetWare 6.5 (32bit) server with 4GB of RAM, simply due to the added room for a file-cache. What I don't know is how CPU-bound file-serving I/O is on a Windows Enterprise Server, that's the one area that could keep NetWare 6.5 (32bit) on top. I already know that OES2-Linux out-performs NetWare for NCP traffic, so long as you stay within CPU bounds.
For high-concurrency applications, as far as I know NetWare still wins.
Thursday, May 10, 2007
Editorial: patch Tuesday.
From Slashdot:
http://searchsecurity.techtarget.com/columnItem/0,294698,sid14_gci1254457,00.html
http://it.slashdot.org/article.pl?sid=07/05/10/1652200
In specific, an opinion that Microsoft should get rid of their regularly scheduled patch release and go to opportunistic patch releases. The argument stems from the damage the MS-DNS flaw has caused. Microsoft had a patch for it, why didn't they release it or some such.
He closes with the statement:
Some people quibble about how long it takes MS to come up with a patch after disclosure (responsible or otherwise), but that's not quite relevant to this particular discussion. Because it DOES take a while for the Microsoft patch pipeline to produce production-quality code, doing a staged release schedule like what they do right now makes all the sense in the world. They can do short-cycle patches, but even then it STILL takes weeks to produce a patch.
I've been at this game long enough to have been around for the opportunistic patch schedule Microsoft followed before they started regulating when they released. And let me tell you, having a schedule for these things helps immensely. We know patches from MS come out on Tuesdays, so we've built into our schedule a 'change management' window Tuesday night expressly for that. This is pre-arranged with our users, we don't have to go to them to take their systems down so long as we do it Tuesday night. (As a side note, our NetWare servers also benefit from this time window).
Under the old regime we'd get a hot patch from Microsoft on Wednesday morning. It is a patch that fixes a problem that is being actively exploited. I go to my management and explain the situation, and I'd have to convince them that the pain experienced by not patching exceeds the pain of downtime in order to get a patching window approved. Or I have to wait for the next change-management window to get the code in, which may be too late.
One thing that is exceedingly clear these days is that when patches from MS are released, the black-hat community falls on them with glad cries to reverse engineer them. Once they have the underlying flaw, which may even be disclosed by the reporting party on release-day such as what happened with the Exchange iCal flaw this time around, Bad Stuff can be coded up to exploit vulnerable systems, a new Metasploit plugin developed, all that fun stuff. In short, waiting for a week or so after a patch is released is becoming more and more a vulnerability in and of itself.
Microsoft's claim that doing it on a release schedule increases the patch uptake rate is a very valid one. Because so many of those patches require downtime to apply, patch application has to be built into the IT management environment. Microsoft is getting better about no-reboot patches, Windows 2003 is better than Windows NT ever was, but there is still a ways to go. Until it becomes possible to patch a live system with no downtime, a static release schedule IS the main way to go. An opportunistic schedule practically guarantees that major IT systems (I'm ignoring home systems for this, that's a very different management regime) won't get the patch for several days to weeks after release. The black-hats have been forcing us to ever shorter lags between patch-release and "too bad, you're hacked".
Also, doing is opportunistically may very well mean MORE patches from Microsoft. This month's batch included 19 CVN numbers for 7 patches. Clearly, some patches bundle more than one fix. I approve of this, since it means less patches I have to apply, and the risk of multiple patches stepping on each other is reduced.
Windows is a horrifically complex system. Microsoft has had a very long history with providing security patches, and they've had problems with:
Side note: This morning I booted to the openSUSE partition on my home laptop for the first time in a while. Once it got done parsing the list of updates, I had something like 79 packages to update including a kernel update. Just the Security-flagged patches took 20 minutes to apply and that didn't include the reboot. In contrast, this month's Windows patches took under 5 minutes to apply. But then, I don't have MS Office on my Windows partition.
http://searchsecurity.techtarget.com/columnItem/0,294698,sid14_gci1254457,00.html
http://it.slashdot.org/article.pl?sid=07/05/10/1652200
In specific, an opinion that Microsoft should get rid of their regularly scheduled patch release and go to opportunistic patch releases. The argument stems from the damage the MS-DNS flaw has caused. Microsoft had a patch for it, why didn't they release it or some such.
He closes with the statement:
The value of the predictability of the monthly schedule simply doesn't outweigh the danger to customers posed by the flaws that go unpatched for three or four weeks between cycles.There is a problem with this. I bring to your attention a post on Bugtraq yesterday from iDefense about the Exchange 2000 IMAP vulnerability. I quote the key piece, which is in section 7:
VIII. DISCLOSURE TIMELINENote the times there. The disclosure was done to Microsoft in January, and it was in May before the fix was released. The time spent between 'initial vendor response' and 'coordinated public disclosure' was spent by Microsoft developing a fix, testing the fix, and integrating the fix into the patch release pipeline. This is part of 'responsible disclosure', which is telling the vendor about a problem, and not telling anyone else about it until the vendor has produced a patch.
01/10/2007 Initial vendor notification
01/22/2007 Initial vendor response
05/08/2007 Coordinated public disclosure
Some people quibble about how long it takes MS to come up with a patch after disclosure (responsible or otherwise), but that's not quite relevant to this particular discussion. Because it DOES take a while for the Microsoft patch pipeline to produce production-quality code, doing a staged release schedule like what they do right now makes all the sense in the world. They can do short-cycle patches, but even then it STILL takes weeks to produce a patch.
I've been at this game long enough to have been around for the opportunistic patch schedule Microsoft followed before they started regulating when they released. And let me tell you, having a schedule for these things helps immensely. We know patches from MS come out on Tuesdays, so we've built into our schedule a 'change management' window Tuesday night expressly for that. This is pre-arranged with our users, we don't have to go to them to take their systems down so long as we do it Tuesday night. (As a side note, our NetWare servers also benefit from this time window).
Under the old regime we'd get a hot patch from Microsoft on Wednesday morning. It is a patch that fixes a problem that is being actively exploited. I go to my management and explain the situation, and I'd have to convince them that the pain experienced by not patching exceeds the pain of downtime in order to get a patching window approved. Or I have to wait for the next change-management window to get the code in, which may be too late.
One thing that is exceedingly clear these days is that when patches from MS are released, the black-hat community falls on them with glad cries to reverse engineer them. Once they have the underlying flaw, which may even be disclosed by the reporting party on release-day such as what happened with the Exchange iCal flaw this time around, Bad Stuff can be coded up to exploit vulnerable systems, a new Metasploit plugin developed, all that fun stuff. In short, waiting for a week or so after a patch is released is becoming more and more a vulnerability in and of itself.
Microsoft's claim that doing it on a release schedule increases the patch uptake rate is a very valid one. Because so many of those patches require downtime to apply, patch application has to be built into the IT management environment. Microsoft is getting better about no-reboot patches, Windows 2003 is better than Windows NT ever was, but there is still a ways to go. Until it becomes possible to patch a live system with no downtime, a static release schedule IS the main way to go. An opportunistic schedule practically guarantees that major IT systems (I'm ignoring home systems for this, that's a very different management regime) won't get the patch for several days to weeks after release. The black-hats have been forcing us to ever shorter lags between patch-release and "too bad, you're hacked".
Also, doing is opportunistically may very well mean MORE patches from Microsoft. This month's batch included 19 CVN numbers for 7 patches. Clearly, some patches bundle more than one fix. I approve of this, since it means less patches I have to apply, and the risk of multiple patches stepping on each other is reduced.
Windows is a horrifically complex system. Microsoft has had a very long history with providing security patches, and they've had problems with:
- Patch order
- Service Packs removing in place patches
- Patches applied simultaneously stepping on each other
- Patches not applying the way they were intended
- Feature or bug regressions
- Patches causing problems in seemingly unrelated programs
- Patches changing 'undocumented behavior' exploited by legitimate 3rd party applications (and sometimes, other Microsoft applications)
Side note: This morning I booted to the openSUSE partition on my home laptop for the first time in a while. Once it got done parsing the list of updates, I had something like 79 packages to update including a kernel update. Just the Security-flagged patches took 20 minutes to apply and that didn't include the reboot. In contrast, this month's Windows patches took under 5 minutes to apply. But then, I don't have MS Office on my Windows partition.
Tuesday, May 01, 2007
Microsoft permissions
It is clear our helpdesks are very used to Novell/NetWare style permissions. Over the years they've gotten used to calling up one of us, having us add someone to a group, and bam they're in! Now that we're doing a bit more with Microsoft permissions, they've run into one of the key differences between how Novell and Microsoft handle permissions.
In MS-land, on login you get an access token with all of your group memberships on it. If you add someone to a group, they have to log out and log in again to refresh that token before they can gain access to the resource.
In Novell-land, every time you access a resource that resource queries the directory service to see if you're in the right groups for access. If you add someone to a group, it takes effect immediately.
Very different expectations! The MS-way may be ultimately more computer resource efficient, but it does come at the cost of user efficiency. This bites us most often when it comes to Exchange. In what we call 'shared mailboxes' we use AD groups to manage access into those. Many times I'll get a call from the helpdesk that a user can't get into a just-created shared mailbox, and this behavior is the reason for it. They're so used to the Novell way of doing it that this seems like an error.
In MS-land, on login you get an access token with all of your group memberships on it. If you add someone to a group, they have to log out and log in again to refresh that token before they can gain access to the resource.
In Novell-land, every time you access a resource that resource queries the directory service to see if you're in the right groups for access. If you add someone to a group, it takes effect immediately.
Very different expectations! The MS-way may be ultimately more computer resource efficient, but it does come at the cost of user efficiency. This bites us most often when it comes to Exchange. In what we call 'shared mailboxes' we use AD groups to manage access into those. Many times I'll get a call from the helpdesk that a user can't get into a just-created shared mailbox, and this behavior is the reason for it. They're so used to the Novell way of doing it that this seems like an error.
Tuesday, April 24, 2007
Migrating a tricky application
We have a line of business application that is used for a small department. Like financial applications, this one is consistently about 4-6 years behind the current trends in application development. Version 9 which was just released a few months ago is finally web based. The version we're running, 8.something, is still based on the same model that Access databases are. I.e., file based databasing.
This particular application is excruciatingly sensitive to oplocks. We've fought this application for years as a result of that. Why is it so sensitive?
Any long time NetWare admin will tell you about PDOXUSRS.NET files. This particular application uses the same kind of access mode. One file is used to mediate who is authorized to access the application as a whole. While users are using the application they keep that file open, and update the file with their application level lock. Okay so far.
The problem comes with oplocks. How it is supposed to work:
One of the things that this vendor has done is decertify NetWare as a valid File Server to store this stuff. This is why I migrated the directory to a Windows 2003 server last night. And even there they had us do a reg-hack to turn off oplock support in Windows. They REALLY do not like oplocks.
Once this app goes web-based, it should help reduce the problems we have with that license file. I hope.
This particular application is excruciatingly sensitive to oplocks. We've fought this application for years as a result of that. Why is it so sensitive?
Any long time NetWare admin will tell you about PDOXUSRS.NET files. This particular application uses the same kind of access mode. One file is used to mediate who is authorized to access the application as a whole. While users are using the application they keep that file open, and update the file with their application level lock. Okay so far.
The problem comes with oplocks. How it is supposed to work:
- Station 100 opens LICENSE.LOG and requests an oplock on it
- Server, seeing no other stations with that file open, grants the oplock.
- Station 100 copies LICENSE.LOG to memory, thus improving access times to it.
- Station 105 opens LICENSE.LOG and requests and oplock on it.
- Server, seeing Station 100 has an oplock on it, tells Station 100 to release its oplock.
- Station 100 writes LICENSE.LOG to the server, and releases its oplock.
- Server tells Station 105 it can open the file, but can't have an oplock.
- Station 105 accesses the file without an oplock.
- Station 100 opens LICENSE.LOG and requests an oplock on it.
- Server, seeing no other stations with that file open, grants the oplock.
- Station 100 copies LICENSE.LOG to memory, thus improving access times to it.
- Station 100 crashes hard. Does not reset its connection to the server, and the Watchdog doesn't scavenge it.
- Station 105 opens LICENSE.LOG and requests an oplock on it.
- Server, seeing station 100 has an oplock on it, tells Station 100 to release its oplock.
- Station 100 is no longer there.
- Server waits until station 100 releases its oplock, which will be never.
- Station 105 doesn't get its lock. Application will not load for anyone else.
- Server Admin hard clears Station 100.
- Application Admin goes into LICENSE.LOG and cleans up bad entries.
- Application Admin tells all stations running Application to log out and log in again.
One of the things that this vendor has done is decertify NetWare as a valid File Server to store this stuff. This is why I migrated the directory to a Windows 2003 server last night. And even there they had us do a reg-hack to turn off oplock support in Windows. They REALLY do not like oplocks.
Once this app goes web-based, it should help reduce the problems we have with that license file. I hope.
Monday, March 19, 2007
Morning keynote, annoying the slashdot crowd
There was a slide that is guaranteed to annoy the slashdot crowd.
Note the bottom line:
Another thing to note about Mr. Mundie's discussion was OSS. He referred to OSS as, "the university model of development," which further implies that it isn't good for industry. It was a subtle thing, but clearly more of the Microsoft line on this whole deal.
Note the bottom line:
- Reduced risk of deployment
Another thing to note about Mr. Mundie's discussion was OSS. He referred to OSS as, "the university model of development," which further implies that it isn't good for industry. It was a subtle thing, but clearly more of the Microsoft line on this whole deal.
Labels: brainshare, microsoft, novell
Monday, February 12, 2007
Novell, Microsoft, and Xen
Novell put out a press release today.
It turns out that Intel has worked out some drivers for use by Windows inside a Xen paravirtualized container. This is distinct from the 'full virtualization' possible only in conjunction with things like the Intel VT instructions. I expected this to maybe be ready in time for SLES10 SP1, if we were lucky.
This is of great interest to me. I'm running windows on Xen right now, in full mode. Network performance is decidedly poor, though the rest of it works reasonably well. I'd like to run it paravirtualized if at all possible as that runs faster. Unfortunately, the drivers mentioned in the PR aren't generally available.
It turns out that Intel has worked out some drivers for use by Windows inside a Xen paravirtualized container. This is distinct from the 'full virtualization' possible only in conjunction with things like the Intel VT instructions. I expected this to maybe be ready in time for SLES10 SP1, if we were lucky.
This is of great interest to me. I'm running windows on Xen right now, in full mode. Network performance is decidedly poor, though the rest of it works reasonably well. I'd like to run it paravirtualized if at all possible as that runs faster. Unfortunately, the drivers mentioned in the PR aren't generally available.
Labels: microsoft, novell, virtualization
Saturday, January 13, 2007
Huh, HTML and Outlook 2007
So I see on Slashdot that Microsoft has announced that Outlook 2007 will use Word as the HTML rendering engine. This makes sense, because they were planning on making Word the default e-mail editor.
I STILL think this is abomination, but then I've thought that about HTML in e-mail for a while now. The best thing that happened to Outlook was Plain Text mode, which if I remember right was a reaction to the HTML-virus mails of a few years ago. Only rarely do I take my Outlook out of plain-text mode to read a mail, generally because some Helpdesk person sent me a mail with an embedded screen-shot in it that can't be viewed any other way.
According to Slashdot, 'email designers' are up in arms because they lose things like CSS. Yes, it takes mail layout back 1998, but I can't view this as a bad thing. That is a personal view. And a slightly professional one, as I've spent the last three days pouring over spam and all the false-positives generated by a certain famous lingerie company having a sale; a mail that contained about every single bad behavior that folk concerned about privacy worry about. Yeah. E-mail based 'newsletters' that look like a web-page... I can see the attraction, but they make my life harder so I avoid them where possible.
So yeah, Microsoft taking CSS support out of email is something I have faintly good ideas over. It'll break the formatting of existing mails, but I very, very rarely see those anyway.
I STILL think this is abomination, but then I've thought that about HTML in e-mail for a while now. The best thing that happened to Outlook was Plain Text mode, which if I remember right was a reaction to the HTML-virus mails of a few years ago. Only rarely do I take my Outlook out of plain-text mode to read a mail, generally because some Helpdesk person sent me a mail with an embedded screen-shot in it that can't be viewed any other way.
According to Slashdot, 'email designers' are up in arms because they lose things like CSS. Yes, it takes mail layout back 1998, but I can't view this as a bad thing. That is a personal view. And a slightly professional one, as I've spent the last three days pouring over spam and all the false-positives generated by a certain famous lingerie company having a sale; a mail that contained about every single bad behavior that folk concerned about privacy worry about. Yeah. E-mail based 'newsletters' that look like a web-page... I can see the attraction, but they make my life harder so I avoid them where possible.
So yeah, Microsoft taking CSS support out of email is something I have faintly good ideas over. It'll break the formatting of existing mails, but I very, very rarely see those anyway.
Labels: microsoft
Monday, January 01, 2007
Dethroning Exchange
A lot of talk has gone into how to overthrow the Windows lock on the Desktop market. The server market is more fluid, but it STILL dominates that space. Linux and OSX are both making real strides in that space, though Apple's ad campaign focusing on, "Windows is for Work, Mac is for Fun," doesn't exactly improve Mac adoption in the workplace.
There aren't any clear threats to Exchange. The other two big players in the arena, GroupWise, and Lotus Notes, have both been there a long time. Both benefited from what I call, 'the Melissa years defections.' I know for a fact that OldJob stayed with GroupWise precicely because we were still up when Melissa and company nuked most of the Exchange shops in the area.
Melissa introduced the era of the mass-mail worm. The clean up efforts from those worms drove billions of dollars of investment into Exchange recovery tools, Exchange anti-virus tools, and other related technologies. Thanks to that burst of innovation, this is a largely solved problem (given a sufficient investment in 3rd party defensive tools). WWU hasn't had a mass-mail-worm-related Exchange outage since I started here three years ago.
What's also helping is that the mass-mail worm is slowly dying by the side of the road in favor of much more lucrative mails. The current SPAM problem is turning into a sort of global denial-of-service attack against SMTP in general, not just Exchange. Trojan emails that contain images that exploit Windows image handling, not just Outlook's, affect even Pegasus users.
The best defence against the current crudware infecting e-mail these days is to use a non-Windows desktop. If that's not in the cards (it isn't for WWU) then the field opens up much more dramatically. Most larger shops are looking seriously into anti-spam appliances as a load-shedding technology to help their mail-transfer-agent (whatever it is) keep up with legitimate load. Some very minority players in the MTA market only can use appliances, and don't have the option of hooked-in anti-spam software.
The days of viruses and other crud scaring people off of Exchange are long gone. Now the fight has to be taken up on, unfortunately, features and mind-share. In the absence of a scare like Melissa provided, migrations from Exchange to something else will be driven by migration events. Microsoft may be providing just that threshold in the future, as they've said that they will be integrating Exchange in with SharePoint to create the End All Be All of groupware applications. Companies that aren't comfortable with that, or haven't deployed SharePoint for whatever reason may see that as an excuse to jump the Microsoft ship for something else. Unfortunately, it'll be executives looking for an excuse rather than executives seeing much better features in, say, GroupWise.
Exchange isn't as dominant as Windows-on-Desktop is, but its market-share isn't exactly declining the way Windows desktop ownership is (really! It is declining! Minuscule amounts, but it is there!). New deployments of Notes or GroupWise, which is different from migrations, are due largely to geeks or management familiar with either technology requesting it specifically. The default is still Exchange when it comes to a big-boy groupware application. That'll take real time to change.
So, Exchange will be with us a long time. What'll start making the throne wobble is if non-Windows desktops start showing up in great numbers in the workplace. THEN we could see some non-MS groupware application threaten Exchange the way that Mac (and Linux) are threatening the desktop.
There aren't any clear threats to Exchange. The other two big players in the arena, GroupWise, and Lotus Notes, have both been there a long time. Both benefited from what I call, 'the Melissa years defections.' I know for a fact that OldJob stayed with GroupWise precicely because we were still up when Melissa and company nuked most of the Exchange shops in the area.
Melissa introduced the era of the mass-mail worm. The clean up efforts from those worms drove billions of dollars of investment into Exchange recovery tools, Exchange anti-virus tools, and other related technologies. Thanks to that burst of innovation, this is a largely solved problem (given a sufficient investment in 3rd party defensive tools). WWU hasn't had a mass-mail-worm-related Exchange outage since I started here three years ago.
What's also helping is that the mass-mail worm is slowly dying by the side of the road in favor of much more lucrative mails. The current SPAM problem is turning into a sort of global denial-of-service attack against SMTP in general, not just Exchange. Trojan emails that contain images that exploit Windows image handling, not just Outlook's, affect even Pegasus users.
The best defence against the current crudware infecting e-mail these days is to use a non-Windows desktop. If that's not in the cards (it isn't for WWU) then the field opens up much more dramatically. Most larger shops are looking seriously into anti-spam appliances as a load-shedding technology to help their mail-transfer-agent (whatever it is) keep up with legitimate load. Some very minority players in the MTA market only can use appliances, and don't have the option of hooked-in anti-spam software.
The days of viruses and other crud scaring people off of Exchange are long gone. Now the fight has to be taken up on, unfortunately, features and mind-share. In the absence of a scare like Melissa provided, migrations from Exchange to something else will be driven by migration events. Microsoft may be providing just that threshold in the future, as they've said that they will be integrating Exchange in with SharePoint to create the End All Be All of groupware applications. Companies that aren't comfortable with that, or haven't deployed SharePoint for whatever reason may see that as an excuse to jump the Microsoft ship for something else. Unfortunately, it'll be executives looking for an excuse rather than executives seeing much better features in, say, GroupWise.
Exchange isn't as dominant as Windows-on-Desktop is, but its market-share isn't exactly declining the way Windows desktop ownership is (really! It is declining! Minuscule amounts, but it is there!). New deployments of Notes or GroupWise, which is different from migrations, are due largely to geeks or management familiar with either technology requesting it specifically. The default is still Exchange when it comes to a big-boy groupware application. That'll take real time to change.
So, Exchange will be with us a long time. What'll start making the throne wobble is if non-Windows desktops start showing up in great numbers in the workplace. THEN we could see some non-MS groupware application threaten Exchange the way that Mac (and Linux) are threatening the desktop.
Thursday, December 21, 2006
2GB Exchange mailboxes? Owie.
http://slashdot.org/article.pl?sid=06/12/21/1655243
MS Fights GMail with 2GB Exchange Mailboxes
Yeesh. OldJob was on GroupWise, and we didn't have mail quotas in place. The largest mailbox I saw (not including archives) was about 900MB. These days that'd probably translate to a 2.5GB mailbox. So yeah, they can get that big.
When I started here the standard Exchange mailbox settings were set to start complaining when the 30MB line was crossed. We've upped it to 46MB since then. We manage our large users by having a higher tier quota group with much higher limits. That group is currently set to start warning at 200MB. Our largest mailbox right now is 233MB.
The problem with mailboxes that large is, of course, backing it all up. The article goes on to say that Exchang 2007 will have features that will help mitigate that. What I suspect that means is replication to another site, rather than the mail archive features some folk use backup/recovery for.
Setting the max quota to 2GB will result in a LOT more people using email as a filing cabinet. Right now the total size of our Exchange system is around 310GB, which is a direct result of those mail quotas I mentioned above. Additionally, we're backing up around 100GB of .PST files on the Novell cluster; this of course does not include those PST files located on PCs. Taking the breaks off the mail quotas would expand our mail significantly faster than its expanding now. Those folk who legitimately deal with huge files will be less inclined to delete redundant copies of Monster Attachments.
One of the more annyoing problems with just taking the breaks off is how long it'll take to sanity-check a bad mail database. The last time we did a round of that the data files were in the 28-30GB range, and it took about eight hours per mail-store to clean the database files. Exchange could handle that no problem, but that did result in an extensive downtime. Two servers, four large mail-stores, meant that once we started the repair process it was a minimum of 16 hours before everything was back up.
It'll be interesting to see the Exchange 2007 guidance for designing enterprises with that much storage.
MS Fights GMail with 2GB Exchange Mailboxes
Yeesh. OldJob was on GroupWise, and we didn't have mail quotas in place. The largest mailbox I saw (not including archives) was about 900MB. These days that'd probably translate to a 2.5GB mailbox. So yeah, they can get that big.
When I started here the standard Exchange mailbox settings were set to start complaining when the 30MB line was crossed. We've upped it to 46MB since then. We manage our large users by having a higher tier quota group with much higher limits. That group is currently set to start warning at 200MB. Our largest mailbox right now is 233MB.
The problem with mailboxes that large is, of course, backing it all up. The article goes on to say that Exchang 2007 will have features that will help mitigate that. What I suspect that means is replication to another site, rather than the mail archive features some folk use backup/recovery for.
Setting the max quota to 2GB will result in a LOT more people using email as a filing cabinet. Right now the total size of our Exchange system is around 310GB, which is a direct result of those mail quotas I mentioned above. Additionally, we're backing up around 100GB of .PST files on the Novell cluster; this of course does not include those PST files located on PCs. Taking the breaks off the mail quotas would expand our mail significantly faster than its expanding now. Those folk who legitimately deal with huge files will be less inclined to delete redundant copies of Monster Attachments.
One of the more annyoing problems with just taking the breaks off is how long it'll take to sanity-check a bad mail database. The last time we did a round of that the data files were in the 28-30GB range, and it took about eight hours per mail-store to clean the database files. Exchange could handle that no problem, but that did result in an extensive downtime. Two servers, four large mail-stores, meant that once we started the repair process it was a minimum of 16 hours before everything was back up.
It'll be interesting to see the Exchange 2007 guidance for designing enterprises with that much storage.
Friday, November 03, 2006
The Novell/Microsoft compact
I just don't have enough background in the Open Source Software environment to comment knowledgably about how this new deal impacts OSS as a whole. I know there are severe misgivings that Novell has sold out, Novell will go the way of a lot of past Microsoft partners, or Novell will by proxy poison Linux with Microsoft's 'adopt and extend' tendrils. It is not without reason that the OSS community views Microsoft as the poster-company for closed-source development.
Now I'm just speaking for me. Speaking as a person who takes vendor lock-in in my datacenter as a cost of doing business, the prospect of only being able to get Linux from Novell doesn't strike fear into my heart. Novell is already a provider of Operating Systems in my datacenter, so this doesn't change anything. It just means we're a bit less exposed to patent-based lawsuits, and I wasn't terribly concerned about that anyway.
The prospect of increased interoperability between SLES and Windows, and hopefully by proxy eDir on NetWare, is a good one. We have a lot of call to replicate identity information between AD and eDir, which is a role handled through custom software. We can't afford pay-for software like IDM. The possibility of federating identity between the two will make some things a lot easier. GroupWise on AD? Exchange on eDir? Could happen! Wouldn't that be a kick in the pants?
In terms of open source in general, what we do use is generally provided by Solaris not Novell. Part of that is an artifact of the fact that most of the xNIX we're running around here is Solaris, not Linux. We have a couple of SLES servers in production, but again generally speaking the only OSS we're running on those is the OS itself.
Virtualization is another area where we'd see some gains. If those two can work together to make Windows more paravirtualization-friendly, or SLES more windows-in-VM friendly we might actually use it. Right now ESX Server is the only real choice, and that's the way we're going to dance for the time being.
And now we all wait with bated breath for what the heck Red Hat is going to do after all of this.
Tags: novell, microsoft
Now I'm just speaking for me. Speaking as a person who takes vendor lock-in in my datacenter as a cost of doing business, the prospect of only being able to get Linux from Novell doesn't strike fear into my heart. Novell is already a provider of Operating Systems in my datacenter, so this doesn't change anything. It just means we're a bit less exposed to patent-based lawsuits, and I wasn't terribly concerned about that anyway.
The prospect of increased interoperability between SLES and Windows, and hopefully by proxy eDir on NetWare, is a good one. We have a lot of call to replicate identity information between AD and eDir, which is a role handled through custom software. We can't afford pay-for software like IDM. The possibility of federating identity between the two will make some things a lot easier. GroupWise on AD? Exchange on eDir? Could happen! Wouldn't that be a kick in the pants?
In terms of open source in general, what we do use is generally provided by Solaris not Novell. Part of that is an artifact of the fact that most of the xNIX we're running around here is Solaris, not Linux. We have a couple of SLES servers in production, but again generally speaking the only OSS we're running on those is the OS itself.
Virtualization is another area where we'd see some gains. If those two can work together to make Windows more paravirtualization-friendly, or SLES more windows-in-VM friendly we might actually use it. Right now ESX Server is the only real choice, and that's the way we're going to dance for the time being.
And now we all wait with bated breath for what the heck Red Hat is going to do after all of this.
Tags: novell, microsoft
