Friday, July 29, 2005
iPrint musings
The big question is how do you allow people to install iPrinters without hosting the web-page on the iPrint server itself. We're working on that. More testing needs to be done, but this is the latest version of how we're getting it to work.
The Maptool that comes with iPrint creates html pages. They have a background image used as a map, and printers are placed on that map to create a sense of floor-plan. No biggie. The URL that actually performs the install is pretty simple, if a bit big:
href="isinstf.htm?ippPrinterURL=ipp://10.1.2.110/ipp/printername-1&onInstalled=status&onNotInstalled=install"
Clearly, if the "isinstf.htm" page isn't on the same server that's hosting the document, it'll fail. No worries. Grab a copy of it, or just reference the iPrinter server directly:
href="https://iprintserver.wwu.edu/ippdocs/isinstf.htm?ippPrinterURL=ipp://10.1.2.110/ipp/printername-1&onInstalled=status&onNotInstalled=install"
That'll work to a point. But the "isinstf.htm" file itself has relative pathing in it that'll trip you up. Probably more so in a cluster. As it turns out, that file, located in SYS:Apache2/htdocs/ippdocs/isinstf.htm, also contains relative pathing that'll screw things up. So prefix "https://iprintserver.yada.yada/" to all of the links, and it seems to work.
The Maptool that comes with iPrint creates html pages. They have a background image used as a map, and printers are placed on that map to create a sense of floor-plan. No biggie. The URL that actually performs the install is pretty simple, if a bit big:
href="isinstf.htm?ippPrinterURL=ipp://10.1.2.110/ipp/printername-1&onInstalled=status&onNotInstalled=install"
Clearly, if the "isinstf.htm" page isn't on the same server that's hosting the document, it'll fail. No worries. Grab a copy of it, or just reference the iPrinter server directly:
href="https://iprintserver.wwu.edu/ippdocs/isinstf.htm?ippPrinterURL=ipp://10.1.2.110/ipp/printername-1&onInstalled=status&onNotInstalled=install"
That'll work to a point. But the "isinstf.htm" file itself has relative pathing in it that'll trip you up. Probably more so in a cluster. As it turns out, that file, located in SYS:Apache2/htdocs/ippdocs/isinstf.htm, also contains relative pathing that'll screw things up. So prefix "https://iprintserver.yada.yada/" to all of the links, and it seems to work.
Wednesday, July 27, 2005
Quiet time
Most of our major projects are on hold thanks to vacations, and getting a lot done already. Some of the stuff we can't start on until 8/22, the first Monday after the last Summer Session lets out. Others require folk who are out of state at the moment.
In other news:
In other news:
- BI-Query is making us crazier than usual
- Blackboard will probably get a rebuild during break
- Our SAN will need replacing a year earlier than we had planned, thanks to greater than expected usage. 06-07 timeframe
- ResTek really really likes iPrint
- The libC problem is still a problem, even with nwlib6d. Yep, that incident is still open.
Friday, July 22, 2005
In local news
A tunnel across the border up near the Aldergrove/Lynden border crossing has a local connection. It turns out that a coworker of mine toured the US house in question when he was house-hunting several years ago.
Wednesday, July 20, 2005
Cooling followup
Now that it has been over 24 hours since I jiggered things around the blade-rack, things have improved markedly. The exhaust coming off the back of the blade-rack is very noticably cooler than it was before. This makes me a lot happier. We might, might be able to stuff one or two more blade chassis into that rack now.
All I did was move a couple of vent-tiles to right in front of the rack, and replace the front door with a fully perforated door. The previous one was only about 20% perforated. With the old door I noticed from back to front circulation over the top of the existing chassis in the rack. Before the exhaust was dragon-like. Now it is merely rather warm.
All I did was move a couple of vent-tiles to right in front of the rack, and replace the front door with a fully perforated door. The previous one was only about 20% perforated. With the old door I noticed from back to front circulation over the top of the existing chassis in the rack. Before the exhaust was dragon-like. Now it is merely rather warm.
Monday, July 18, 2005
Rack cooling
I just spent some time trying to improve the air-flow around and through our blade-rack. In the process I learned that our vent-tile lifter has wandered away somewhere along the line. I think I managed to improve things a touch.
Our datacenter's air-flow is a little hap-hazard, but our cooling capacity is far enough ahead of heat generation that we don't have hot-spots yet. The blade-rack is by far the #1 heat producer in the room. The SAN disk-packs do not pump out the heat you'd think they would with the power they suck, all that energy gets converted to actual work spinning disks rather than being resisted off in the process of number-crunching. The rise in temp from front to back for the blade rack has to be pushing 25 degrees F.
I moved a couple of vent tiles to right in front of the blade-rack, and that helped some. Then some air-flow testing (thanks to a couple of hairs around 4" in length, they make good free flow testers) inside the rack itself showed that we're getting a bit of back-to-front circulation over the top of the topmost blade chassis. That's not good. Opening the rack door caused the circulation to maintain front-to-back up there, so that told me the front door needed attention. We found another fully perforated front door and replaced it. The circulation went a bit neutral, but at least it wasn't back-to-front like before.
All in all, the work reduced the temp of the back of the rack to 'only' about +15 to +20 degrees. I don't have a thermometer I can use for this, so I'm not certain. But it is a bit cooler back there now.
Our datacenter's air-flow is a little hap-hazard, but our cooling capacity is far enough ahead of heat generation that we don't have hot-spots yet. The blade-rack is by far the #1 heat producer in the room. The SAN disk-packs do not pump out the heat you'd think they would with the power they suck, all that energy gets converted to actual work spinning disks rather than being resisted off in the process of number-crunching. The rise in temp from front to back for the blade rack has to be pushing 25 degrees F.
I moved a couple of vent tiles to right in front of the blade-rack, and that helped some. Then some air-flow testing (thanks to a couple of hairs around 4" in length, they make good free flow testers) inside the rack itself showed that we're getting a bit of back-to-front circulation over the top of the topmost blade chassis. That's not good. Opening the rack door caused the circulation to maintain front-to-back up there, so that told me the front door needed attention. We found another fully perforated front door and replaced it. The circulation went a bit neutral, but at least it wasn't back-to-front like before.
All in all, the work reduced the temp of the back of the rack to 'only' about +15 to +20 degrees. I don't have a thermometer I can use for this, so I'm not certain. But it is a bit cooler back there now.
Thursday, July 14, 2005
Hmmmm
dShield.com
Looks like there has been a real uptick in scanning for VNC servers. I wonder what prompted that?
Looks like there has been a real uptick in scanning for VNC servers. I wonder what prompted that?
Wednesday, July 13, 2005
New phish
I got a new kind of eBay phish today. Apparently I can become a 'titanium power seller', even though I don't even have an eBay account! I haven't looked at it in rendered HTML form, but the mail includes some items that are interesting.
eBay Help - which goes to http://mail.yahoo.com/config/login?/xxxxxxxxxxxxxxxx instead. Riiiiight.
With this mark, you can be confident that you are transactingAnd so it goes. Along with this gem.
with an experienced eBay user who has proven that they're committed
to customer satisfaction.
Joining this grup is a simple two- step process.Grup?
http://mail.gsmmaroc.ma/.Update/eBay-Account/Index.htmlHardly and eBay account. Considering that the image they use is located somewhere else:
http://www.soldsmart.biz/Images/ti_powerseller.gifAnyway, this is a deviation from the standard, 'please verify your information or we'll cut you off,' phish.
eBay Help - which goes to http://mail.yahoo.com/config/login?/xxxxxxxxxxxxxxxx instead. Riiiiight.
NetWare and the 4GB limit
There is a TID on this one. In short, NW65 can address up to 64GB of memory, and NW6 is limited to 4GB. NW65 can't do as much as you'd think with that access.
Think of it like EMS back in the DOS days. In order to address memory over the limit, the requested info has to be paged below the limit before it can be handled. Just like a page-file, but with faster access. This can be done invisibly to the user, but it is an expensive operation all things considered.
NW65 considers memory above 4GB as virtual memory. NSS can use this memory as well, though due to the speed issue it only caches files larger than 128K. Protected memory spaces can use this memory since they can access VM. If you have whonking huge applications that need lots of RAM, you'd better run them in a protected memory space to gain the most benefit.
When the 64-bit version gets released, the memory limits go into the stratosphere again.
But since the reason we need to add RAM is to make the NSS cache larger, we're just FINE with this. We're going to end up with 5gb in the student-side cluster nodes due to how things work. This'll give us p-l-e-n-t-y of room to grow in the next year.
Think of it like EMS back in the DOS days. In order to address memory over the limit, the requested info has to be paged below the limit before it can be handled. Just like a page-file, but with faster access. This can be done invisibly to the user, but it is an expensive operation all things considered.
NW65 considers memory above 4GB as virtual memory. NSS can use this memory as well, though due to the speed issue it only caches files larger than 128K. Protected memory spaces can use this memory since they can access VM. If you have whonking huge applications that need lots of RAM, you'd better run them in a protected memory space to gain the most benefit.
When the 64-bit version gets released, the memory limits go into the stratosphere again.
But since the reason we need to add RAM is to make the NSS cache larger, we're just FINE with this. We're going to end up with 5gb in the student-side cluster nodes due to how things work. This'll give us p-l-e-n-t-y of room to grow in the next year.
Tuesday, July 12, 2005
New MBSA released
MBSA 2.0 came out. I hadn't heard. Looks like mostly improved UI.
Monday, July 11, 2005
More migrations
I just finished migrating the first node of the cluster over to a blade-server. That went surprisingly well! I'm very happy.
The first step was the same as it was with the NDS servers earlier this year. Install the new server into its own tree, take a config.txt from the old server for reference, run the Migration Wizard, fiddle things to get the identity settled down. With the NDS servers this was pretty much it, since all they ran was NDS (iManager was on the one server that didn't get migrated) and didn't need any additional work.
The second step was installing the apps. NetStorage needed to be added in and configured. And that required all the supporting Apache and Tomcat bits. Thrown in OpenSSH and FTP, with a bit of iManager, and a dash of apps that the config.txt document said were installed on the old machine, and I had it all in. NetStorage got configured right.
The third step was to get all the cluster scripts we use into their correct places, and all the supporting config files where they needed to go. Since we don't put NetStorage into httpd.conf, rather into its own file, that had to be copied. As did the config file for the myweb service. Then the ncf-files that kick off both web-servers (myfiles and myweb), as well as the trio of FTP/SFTP services. That went pretty well.
Then came the Very Scary process of allowing the new server to see the cluster volumes on the SAN. That took a bit of work. The Fibre switches are zoned by port rather than WWN, which made that step easier. The EVA then needed to be told to permit the new WWN to see the Novell cluster volumes. That took a bit of poking, but in the end it was pretty simple. I hit the commit button on the EVA interface, scanned for new devices, and then loaded NCS. It saw volumes! So I rebooted, and it entered the cluster just peachy.
From begining to end, the whole process took maybe five hours. And that includes the initial imaging of the blade server, and the install of NetWare to the blade. I'm pumped.
AND Novell released nwlib6d today. They promised me that it fixes the nxcreatepathcontext problem I've been having for just over a year now with the myweb and sftp services. I've heard that one before. It's in testing on the student side of the cluster. Early indications are good, but we'll have to see how it lasts over a couple of weeks.
The first step was the same as it was with the NDS servers earlier this year. Install the new server into its own tree, take a config.txt from the old server for reference, run the Migration Wizard, fiddle things to get the identity settled down. With the NDS servers this was pretty much it, since all they ran was NDS (iManager was on the one server that didn't get migrated) and didn't need any additional work.
The second step was installing the apps. NetStorage needed to be added in and configured. And that required all the supporting Apache and Tomcat bits. Thrown in OpenSSH and FTP, with a bit of iManager, and a dash of apps that the config.txt document said were installed on the old machine, and I had it all in. NetStorage got configured right.
The third step was to get all the cluster scripts we use into their correct places, and all the supporting config files where they needed to go. Since we don't put NetStorage into httpd.conf, rather into its own file, that had to be copied. As did the config file for the myweb service. Then the ncf-files that kick off both web-servers (myfiles and myweb), as well as the trio of FTP/SFTP services. That went pretty well.
Then came the Very Scary process of allowing the new server to see the cluster volumes on the SAN. That took a bit of work. The Fibre switches are zoned by port rather than WWN, which made that step easier. The EVA then needed to be told to permit the new WWN to see the Novell cluster volumes. That took a bit of poking, but in the end it was pretty simple. I hit the commit button on the EVA interface, scanned for new devices, and then loaded NCS. It saw volumes! So I rebooted, and it entered the cluster just peachy.
From begining to end, the whole process took maybe five hours. And that includes the initial imaging of the blade server, and the install of NetWare to the blade. I'm pumped.
AND Novell released nwlib6d today. They promised me that it fixes the nxcreatepathcontext problem I've been having for just over a year now with the myweb and sftp services. I've heard that one before. It's in testing on the student side of the cluster. Early indications are good, but we'll have to see how it lasts over a couple of weeks.
Friday, July 08, 2005
Cleaning servers
We had another round of hacked servers in the past few weeks. Strangely enough, the rootkit used was very similar to the one used around the time of this cryptic post. Since I cleaned that round the last time, I was able to figure out this lated round pretty quickly.
What hit us was a variant of the Hacker Defender toolkit, but localized into French (I think). I was able to manually remove the puppy before other folks around here were able to locate a remover program. Just because I like sharing, this is what I discovered.
The toolkit creates three services:
They also created another folder in "C:\System Volume Information\tracking\", which you normally can't get at from explorer. You have to set permissions on the System Volume Information folder in order to get into it. In that folder was various recon and spreading tools, as well as pwdump4 for dumping the local SAM database. They didn't get to anything with any real data in it, so our passwords are safe. Plus, our local admin passwords are longer than 16 characters, so rainbow-tables won't work.
In order to get things back, I set the SYSTEM user to Deny Access for the following directories:
As a result of this, the rootkit wasn't able to initialize and the hidden registry keys were visible again.
None of the above stuff is googleable that I've found, but I've seen it before. So now when other people get hacked by this crew, they'll have a resource.
What hit us was a variant of the Hacker Defender toolkit, but localized into French (I think). I was able to manually remove the puppy before other folks around here were able to locate a remover program. Just because I like sharing, this is what I discovered.
The toolkit creates three services:
- dcrssdrv, hidden, launches %windowsdir%\system32\spool\tracking\in\dcrssdrv.sys
- mmsm, visible, launches %windowsdir%\system32\spool\tracking\in\mmsm.exe
- Description: Optimize transfert between CPU&RAM
- Display Name: Memory Manager
- tcp-ip_port_analyzer, hidden, launches %windowsdir\system32\spool\tracking\in\dcrs.exe dcrs.ini
- Description: Manage all Netbios Connections
- Display Name: Tcp-ip Netbios Monitoring
They also created another folder in "C:\System Volume Information\tracking\", which you normally can't get at from explorer. You have to set permissions on the System Volume Information folder in order to get into it. In that folder was various recon and spreading tools, as well as pwdump4 for dumping the local SAM database. They didn't get to anything with any real data in it, so our passwords are safe. Plus, our local admin passwords are longer than 16 characters, so rainbow-tables won't work.
In order to get things back, I set the SYSTEM user to Deny Access for the following directories:
- %windowsdir%\system32\spool\tracking\
- c:\System Volume Information\tracking\
As a result of this, the rootkit wasn't able to initialize and the hidden registry keys were visible again.
- HKLM\System\CurrentControlSet\Services\dcrssdrv
- HKLM\System\CurrentControlSet\Services\mmsm
- HKLM\System\CurrentControlSet\Services\tcp-ip_port_analyzer
- HKLM\System\ControlSet002\Services\dcrssdrv
- HKLM\System\ControlSet002\Services\mmsm
- HKLM\System\ControlSet002\Services\tcp-ip_port_analyzer
- HKLM\System\CurrentControlSet\Control\SafeBoot\Minimal\Tcp-ip_port_analyzer
- HKLM\System\CurrentControlSet\Control\SafeBoot\Network\Tcp-ip_port_analyzer
- HKLM\System\ControlSet002\Control\SafeBoot\Minimal\Tcp-ip_port_analyzer
- HKLM\System\ControlSet002\Control\SafeBoot\Network\Tcp-ip_port_analyzer
None of the above stuff is googleable that I've found, but I've seen it before. So now when other people get hacked by this crew, they'll have a resource.
Thursday, July 07, 2005
Returning
I'm back!
Things did not melt down in my absence. No swarms of e-mail to thumb through. No voice-mail waiting for me to fix problems two weeks old.
But things continue to change. The AD controllers, all but one, have been migrated to blades. From the spew of boxes in the computer room, it looks like the new Titan machine is here. There is a swank new PowerEdge 2850 sitting on a table for a department's off-the-shelf something-or-other; it'll get racked in a week or so when we make space for it. The Blackboard server needs a nuke-from-orbit rebuild, which will probably get done in the dead-weeks between the end of summer session and begining of fall session.
The next big project is migrating the Faculty/Staff side of the cluster to blades. Shouldn't take all that much time, but running the fibre will be the big part.
Things did not melt down in my absence. No swarms of e-mail to thumb through. No voice-mail waiting for me to fix problems two weeks old.
But things continue to change. The AD controllers, all but one, have been migrated to blades. From the spew of boxes in the computer room, it looks like the new Titan machine is here. There is a swank new PowerEdge 2850 sitting on a table for a department's off-the-shelf something-or-other; it'll get racked in a week or so when we make space for it. The Blackboard server needs a nuke-from-orbit rebuild, which will probably get done in the dead-weeks between the end of summer session and begining of fall session.
The next big project is migrating the Faculty/Staff side of the cluster to blades. Shouldn't take all that much time, but running the fibre will be the big part.
